Max level shown:
1 [ Aug 30 19:40:33 Enabled. ]
2 [ Aug 30 19:40:33 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/sled-agent/sled-agent run /opt/oxide/sled-agent/pkg/config.toml &"). ]
3 [ Aug 30 19:40:33 Method "start" exited with status 0. ]
4 [ Aug 30 19:40:33 Rereading configuration. ]
5 [ Aug 30 19:40:33 No 'refresh' method defined. Treating as :true. ]
6 note: configured to log to "/dev/stdout"
72024-08-30T19:40:33.986ZINFOSledAgent: Ensuring contract reaper thread file = illumos-utils/src/running_zone.rs:113
82024-08-30T19:40:33.987ZINFOSledAgent (BootstrapAgentStartup): finding links ["net0", "net1"] file = sled-agent/src/bootstrap/pre_server.rs:255
92024-08-30T19:40:34.042ZINFOSledAgent (BootstrapAgentStartup): Starting mg-ddm service addr-objs = [AddrObject { interface: "net0", name: "ll" }, AddrObject { interface: "net1", name: "ll" }] file = sled-agent/src/bootstrap/pre_server.rs:262
102024-08-30T19:40:34.042ZINFOSledAgent (BootstrapAgentStartup): Importing mg-ddm service file = sled-agent/src/bootstrap/maghemite.rs:42 path = /opt/oxide/mg-ddm/pkg/ddm/manifest.xml
112024-08-30T19:40:34.157ZINFOSledAgent (BootstrapAgentStartup): Setting mg-ddm interfaces file = sled-agent/src/bootstrap/maghemite.rs:50 interfaces = ("net0/ll" "net1/ll")
122024-08-30T19:40:34.166ZINFOSledAgent (BootstrapAgentStartup): Enabling mg-ddm service file = sled-agent/src/bootstrap/maghemite.rs:56
132024-08-30T19:40:34.174ZINFOSledAgent (BootstrapAgentStartup): Ensuring zfs key directory exists file = sled-agent/src/bootstrap/pre_server.rs:272 path = /var/run/oxide/
142024-08-30T19:40:34.775ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fdb0:18c0:4d0d:9fb2::, width: 64 }
152024-08-30T19:40:34.916ZINFOSledAgent: Starting KeyManager file = sled-agent/src/long_running_tasks.rs:126
162024-08-30T19:40:34.916ZINFOSledAgent: Starting StorageManager file = sled-agent/src/long_running_tasks.rs:138
172024-08-30T19:40:34.916ZINFOSledAgent: Starting StorageMonitor file = sled-agent/src/long_running_tasks.rs:151
182024-08-30T19:40:34.916ZINFOSledAgent: Starting HardwareManager file = sled-agent/src/long_running_tasks.rs:173 nongimlet_observed_disks = [] sled_mode = Scrimlet { asic: SoftNpuZone }
192024-08-30T19:40:34.916ZINFOSledAgent (DumpSetup-worker): DumpSetup poll loop started. file = sled-agent/src/dump_setup.rs:520
202024-08-30T19:40:34.916ZINFOSledAgent (HardwareManager): Creating HardwareManager file = sled-hardware/src/illumos/mod.rs:716
212024-08-30T19:40:34.919ZINFOSledAgent (HardwareManager): Generated i86pc baseboard Pc { identifier: "buskin", model: "i86pc" } file = sled-hardware/src/illumos/mod.rs:623
222024-08-30T19:40:34.919ZWARNSledAgent (HardwareManager): Device is not a Gimlet (i86pc), proceeding with null hardware view file = sled-hardware/src/illumos/mod.rs:765
232024-08-30T19:40:34.919ZINFOSledAgent: Starting HardwareMonitor file = sled-agent/src/long_running_tasks.rs:187
242024-08-30T19:40:34.919ZINFOSledAgent (HardwareManager): Monitoring for hardware updates file = sled-hardware/src/illumos/mod.rs:824
252024-08-30T19:40:34.919ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/m2_0.vdev
262024-08-30T19:40:34.919ZINFOSledAgent (HardwareMonitor): Checking current full hardware snapshot disks = {} file = sled-agent/src/hardware_monitor.rs:271 underlay_network_info = None
272024-08-30T19:40:34.919ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:451
282024-08-30T19:40:34.919ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
292024-08-30T19:40:34.919ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } path = /scratch/m2_0.vdev
302024-08-30T19:40:34.930ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/m2_0.vdev file = sled-hardware/src/disk.rs:360
312024-08-30T19:40:34.930ZINFOSledAgent (StorageResources): Formatting zpool with generated ID file = sled-hardware/src/disk.rs:371 id = fc9e5708-cd50-4950-8336-db011df04e25 (zpool)
322024-08-30T19:40:35.036ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:174 zpool = ZpoolName { id: fc9e5708-cd50-4950-8336-db011df04e25 (zpool), kind: Internal }
332024-08-30T19:40:35.452ZINFOSledAgent (StorageManager): Received DetectedRawDisksChanged { raw_disks: {}, tx: ... } file = sled-storage/src/manager.rs:451
342024-08-30T19:40:35.452ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/m2_1.vdev
352024-08-30T19:40:35.452ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
362024-08-30T19:40:35.452ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
372024-08-30T19:40:35.452ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: fc9e5708-cd50-4950-8336-db011df04e25 (zpool), kind: Internal } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
382024-08-30T19:40:35.452ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:451
392024-08-30T19:40:35.452ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
402024-08-30T19:40:35.452ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
412024-08-30T19:40:35.452ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
422024-08-30T19:40:35.452ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" } path = /scratch/m2_1.vdev
432024-08-30T19:40:35.461ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/m2_1.vdev file = sled-hardware/src/disk.rs:360
442024-08-30T19:40:35.461ZINFOSledAgent (StorageResources): Formatting zpool with generated ID file = sled-hardware/src/disk.rs:371 id = 17afe9d4-47cb-4799-8d3d-ad69badedc16 (zpool)
452024-08-30T19:40:35.549ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:174 zpool = ZpoolName { id: 17afe9d4-47cb-4799-8d3d-ad69badedc16 (zpool), kind: Internal }
462024-08-30T19:40:35.952ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_0.vdev
472024-08-30T19:40:35.952ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: fc9e5708-cd50-4950-8336-db011df04e25 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 17afe9d4-47cb-4799-8d3d-ad69badedc16 (zpool), kind: Internal } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
482024-08-30T19:40:35.952ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
492024-08-30T19:40:35.952ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
502024-08-30T19:40:35.952ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:451
512024-08-30T19:40:35.952ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
522024-08-30T19:40:35.952ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_1.vdev
532024-08-30T19:40:35.952ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: fc9e5708-cd50-4950-8336-db011df04e25 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 17afe9d4-47cb-4799-8d3d-ad69badedc16 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
542024-08-30T19:40:35.952ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
552024-08-30T19:40:35.952ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
562024-08-30T19:40:35.952ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:451
572024-08-30T19:40:35.952ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
582024-08-30T19:40:35.952ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_2.vdev
592024-08-30T19:40:35.952ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: fc9e5708-cd50-4950-8336-db011df04e25 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 17afe9d4-47cb-4799-8d3d-ad69badedc16 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
602024-08-30T19:40:35.952ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
612024-08-30T19:40:35.952ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
622024-08-30T19:40:35.952ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:451
632024-08-30T19:40:35.952ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
642024-08-30T19:40:35.952ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_3.vdev
652024-08-30T19:40:35.952ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: fc9e5708-cd50-4950-8336-db011df04e25 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 17afe9d4-47cb-4799-8d3d-ad69badedc16 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
662024-08-30T19:40:35.952ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
672024-08-30T19:40:35.952ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
682024-08-30T19:40:35.952ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:451
692024-08-30T19:40:35.952ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
702024-08-30T19:40:35.952ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_4.vdev
712024-08-30T19:40:35.952ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: fc9e5708-cd50-4950-8336-db011df04e25 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 17afe9d4-47cb-4799-8d3d-ad69badedc16 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
722024-08-30T19:40:35.952ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
732024-08-30T19:40:35.952ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
742024-08-30T19:40:35.952ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:451
752024-08-30T19:40:35.952ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
762024-08-30T19:40:35.952ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_5.vdev
772024-08-30T19:40:35.952ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: fc9e5708-cd50-4950-8336-db011df04e25 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 17afe9d4-47cb-4799-8d3d-ad69badedc16 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
782024-08-30T19:40:35.952ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:451
792024-08-30T19:40:35.952ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
802024-08-30T19:40:35.952ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
812024-08-30T19:40:35.952ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
822024-08-30T19:40:35.952ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_6.vdev
832024-08-30T19:40:35.952ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: fc9e5708-cd50-4950-8336-db011df04e25 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 17afe9d4-47cb-4799-8d3d-ad69badedc16 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
842024-08-30T19:40:35.952ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
852024-08-30T19:40:35.952ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
862024-08-30T19:40:35.952ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:451
872024-08-30T19:40:35.952ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
882024-08-30T19:40:35.952ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_7.vdev
892024-08-30T19:40:35.952ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:451
902024-08-30T19:40:35.952ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
912024-08-30T19:40:35.952ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: fc9e5708-cd50-4950-8336-db011df04e25 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 17afe9d4-47cb-4799-8d3d-ad69badedc16 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
922024-08-30T19:40:35.953ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
932024-08-30T19:40:35.953ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
942024-08-30T19:40:35.953ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_8.vdev
952024-08-30T19:40:35.953ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:451
962024-08-30T19:40:35.953ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: fc9e5708-cd50-4950-8336-db011df04e25 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 17afe9d4-47cb-4799-8d3d-ad69badedc16 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
972024-08-30T19:40:35.953ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
982024-08-30T19:40:35.953ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
992024-08-30T19:40:35.953ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
1002024-08-30T19:40:35.953ZINFOSledAgent: Waiting for boot disk file = sled-agent/src/long_running_tasks.rs:97
1012024-08-30T19:40:35.953ZINFOSledAgent: Found boot disk DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } file = sled-agent/src/long_running_tasks.rs:99
1022024-08-30T19:40:35.953ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: fc9e5708-cd50-4950-8336-db011df04e25 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 17afe9d4-47cb-4799-8d3d-ad69badedc16 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
1032024-08-30T19:40:35.953ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
1042024-08-30T19:40:35.953ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
1052024-08-30T19:40:35.953ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
1062024-08-30T19:40:35.953ZINFOSledAgent: Starting Bootstore file = sled-agent/src/long_running_tasks.rs:211
1072024-08-30T19:40:35.953ZINFOSledAgent (bootstore): No ledger in /pool/int/fc9e5708-cd50-4950-8336-db011df04e25/cluster/bootstore-fsm-state.json file = common/src/ledger.rs:188 peer_id = pc-buskin-i86pc
1082024-08-30T19:40:35.953ZINFOSledAgent (bootstore): No ledger in /pool/int/17afe9d4-47cb-4799-8d3d-ad69badedc16/cluster/bootstore-fsm-state.json file = common/src/ledger.rs:188 peer_id = pc-buskin-i86pc
1092024-08-30T19:40:35.953ZINFOSledAgent (bootstore): No ledger found. Loading Fsm::State as Uninitialized file = bootstore/src/schemes/v0/storage.rs:83 peer_id = pc-buskin-i86pc
1102024-08-30T19:40:35.953ZINFOSledAgent (bootstore): No ledger in /pool/int/fc9e5708-cd50-4950-8336-db011df04e25/cluster/bootstore-network-config.json file = common/src/ledger.rs:188 peer_id = pc-buskin-i86pc
1112024-08-30T19:40:35.953ZINFOSledAgent (bootstore): No ledger in /pool/int/17afe9d4-47cb-4799-8d3d-ad69badedc16/cluster/bootstore-network-config.json file = common/src/ledger.rs:188 peer_id = pc-buskin-i86pc
1122024-08-30T19:40:35.953ZINFOSledAgent (bootstore): No ledger found for network config file = bootstore/src/schemes/v0/storage.rs:146 peer_id = pc-buskin-i86pc
1132024-08-30T19:40:35.953ZINFOSledAgent: Starting Bootstore DDMD poller file = sled-agent/src/long_running_tasks.rs:216
1142024-08-30T19:40:35.953ZINFOSledAgent: Starting ZoneBundler related tasks file = sled-agent/src/long_running_tasks.rs:231
1152024-08-30T19:40:35.963ZINFOSledAgent (ServiceManager): Creating ServiceManager file = sled-agent/src/services.rs:730
1162024-08-30T19:40:35.963ZINFOSledAgent (auto-cleanup-task): top of bundle cleanup loop file = sled-agent/src/zone_bundle.rs:186 next_cleanup = Instant { tv_sec: 1322, tv_nsec: 632655792 } time_to_next_cleanup = 599.99994608s
1172024-08-30T19:40:35.963ZINFOSledAgent (ServiceManager): Ensuring scrimlet services (enabling services) file = sled-agent/src/services.rs:3866
1182024-08-30T19:40:35.963ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
1192024-08-30T19:40:35.963ZINFOSledAgent (BootstrapAgentStartup): No ledger in /pool/int/fc9e5708-cd50-4950-8336-db011df04e25/config/sled-agent-request.json file = common/src/ledger.rs:188
1202024-08-30T19:40:35.966ZINFOSledAgent (BootstrapAgentStartup): No ledger in /pool/int/17afe9d4-47cb-4799-8d3d-ad69badedc16/config/sled-agent-request.json file = common/src/ledger.rs:188
1212024-08-30T19:40:35.967ZINFOSledAgent (dropshot (BootstrapAgent)): listening file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:205 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80
1222024-08-30T19:40:35.967ZINFOSledAgent (SledAgentSprocketsServer): Started listening file = sled-agent/src/bootstrap/sprockets_server.rs:44 local_addr = [fdb0:18c0:4d0d:9fb2::1]:12346
1232024-08-30T19:40:35.967ZINFOSledAgent (RSS): Injecting RSS configuration: RackInitializeRequest { trust_quorum_peers: None, bootstrap_discovery: OnlyOurs, ntp_servers: [ "0.pool.ntp.org", ], dns_servers: [ 1.1.1.1, 9.9.9.9, ], internal_services_ip_pool_ranges: [ V4( Ipv4Range { first: 10.151.1.164, last: 10.151.1.173, }, ), ], external_dns_ips: [ 10.151.1.164, 10.151.1.165, ], external_dns_zone_name: "oxide.test", external_certificates: "<redacted>", recovery_silo: RecoverySiloConfig { silo_name: Name( "recovery", ), user_name: UserId( "recovery", ), user_password_hash: NewPasswordHash( PasswordHashString { string: "$argon2id$v=19$m=98304,t=13,p=1$RUlWc0ZxaHo0WFdrN0N6ZQ$S8p52j85GPvMhR/ek3GL0el/oProgTwWpHJZ8lsQQoY", encoding: B64, }, ), }, rack_network_config: RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, infra_ip_first: 10.151.1.174, infra_ip_last: 10.151.1.174, ports: [ PortConfigV2 { routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), nexthop: 10.151.1.100, vlan_id: None, local_pref: None, }, ], addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.1.174, width: 24, }, ), vlan_id: None, }, ], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: None, bgp_peers: [], autoneg: false, lldp: None, }, ], bgp: [], bfd: [], }, allowed_source_ips: Any, } file = sled-agent/src/rack_setup/service.rs:1053
1242024-08-30T19:40:35.967ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = internal-dns/src/resolver.rs:73
1252024-08-30T19:40:35.967ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
1262024-08-30T19:40:35.967ZINFOSledAgent (RSS): No ledger in /pool/int/fc9e5708-cd50-4950-8336-db011df04e25/config/rss-plan-completed.marker file = common/src/ledger.rs:188
1272024-08-30T19:40:35.967ZINFOSledAgent (RSS): No ledger in /pool/int/17afe9d4-47cb-4799-8d3d-ad69badedc16/config/rss-plan-completed.marker file = common/src/ledger.rs:188
1282024-08-30T19:40:35.967ZINFOSledAgent (RSS): RSS configuration has not been fully applied yet file = sled-agent/src/rack_setup/service.rs:1113
1292024-08-30T19:40:35.967ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
1302024-08-30T19:40:35.967ZINFOSledAgent (RSS): No ledger in /pool/int/fc9e5708-cd50-4950-8336-db011df04e25/config/rss-sled-plan.json file = common/src/ledger.rs:188
1312024-08-30T19:40:35.967ZINFOSledAgent (RSS): No ledger in /pool/int/17afe9d4-47cb-4799-8d3d-ad69badedc16/config/rss-sled-plan.json file = common/src/ledger.rs:188
1322024-08-30T19:40:35.967ZINFOSledAgent (RSS): Creating new allocation plan file = sled-agent/src/rack_setup/service.rs:1154
1332024-08-30T19:40:35.967ZINFOSledAgent (RSS): Serializing plan file = sled-agent/src/rack_setup/plan/sled.rs:155
1342024-08-30T19:40:35.967ZINFOSledAgent (RSS): Creating plan for the sled at fdb0:18c0:4d0d:9fb2::1 file = sled-agent/src/rack_setup/plan/sled.rs:128
1352024-08-30T19:40:35.967ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
1362024-08-30T19:40:35.967ZINFOSledAgent (RSS): Writing ledger to /pool/int/fc9e5708-cd50-4950-8336-db011df04e25/config/.rss-sled-plan.json.tmp file = common/src/ledger.rs:199
1372024-08-30T19:40:35.967ZINFOSledAgent (RSS): Writing ledger to /pool/int/17afe9d4-47cb-4799-8d3d-ad69badedc16/config/.rss-sled-plan.json.tmp file = common/src/ledger.rs:199
1382024-08-30T19:40:35.968ZINFOSledAgent (RSS): Sled plan written to storage: Plan { rack_id: 27cfef72-d05b-490f-ac92-5c7cad830c2e, sleds: { [fdb0:18c0:4d0d:9fb2::1]:12346: StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: 289c9d10-fa69-4638-aac7-3cfc6b4e23be, rack_id: 27cfef72-d05b-490f-ac92-5c7cad830c2e, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64, }, }, }, }, }, config: RackInitializeRequest { trust_quorum_peers: None, bootstrap_discovery: OnlyOurs, ntp_servers: [ "0.pool.ntp.org", ], dns_servers: [ 1.1.1.1, 9.9.9.9, ], internal_services_ip_pool_ranges: [ V4( Ipv4Range { first: 10.151.1.164, last: 10.151.1.173, }, ), ], external_dns_ips: [ 10.151.1.164, 10.151.1.165, ], external_dns_zone_name: "oxide.test", external_certificates: "<redacted>", recovery_silo: RecoverySiloConfig { silo_name: Name( "recovery", ), user_name: UserId( "recovery", ), user_password_hash: NewPasswordHash( PasswordHashString { string: "$argon2id$v=19$m=98304,t=13,p=1$RUlWc0ZxaHo0WFdrN0N6ZQ$S8p52j85GPvMhR/ek3GL0el/oProgTwWpHJZ8lsQQoY", encoding: B64, }, ), }, rack_network_config: RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, infra_ip_first: 10.151.1.174, infra_ip_last: 10.151.1.174, ports: [ PortConfigV2 { routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), nexthop: 10.151.1.100, vlan_id: None, local_pref: None, }, ], addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.1.174, width: 24, }, ), vlan_id: None, }, ], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: None, bgp_peers: [], autoneg: false, lldp: None, }, ], bgp: [], bfd: [], }, allowed_source_ips: Any, }, } file = sled-agent/src/rack_setup/plan/sled.rs:175
1392024-08-30T19:40:35.968ZINFOSledAgent (RSS): Writing Rack Network Configuration to bootstore file = sled-agent/src/rack_setup/service.rs:1187
1402024-08-30T19:40:35.968ZINFOSledAgent (bootstore): Attempting to update network config with generation: 1, current_generation: 0 file = bootstore/src/schemes/v0/peer.rs:564 peer_id = pc-buskin-i86pc
1412024-08-30T19:40:35.968ZINFOSledAgent (bootstore): Writing ledger to /pool/int/fc9e5708-cd50-4950-8336-db011df04e25/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-buskin-i86pc
1422024-08-30T19:40:35.968ZINFOSledAgent (bootstore): Writing ledger to /pool/int/17afe9d4-47cb-4799-8d3d-ad69badedc16/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-buskin-i86pc
1432024-08-30T19:40:35.968ZINFOSledAgent (bootstore): Broadcasting network config with generation 1 file = bootstore/src/schemes/v0/peer.rs:648 peer_id = pc-buskin-i86pc
1442024-08-30T19:40:35.968ZINFOSledAgent (BootstrapAgentRssHandler): Received initialization request from RSS file = sled-agent/src/bootstrap/rss_handle.rs:217 request = StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: 289c9d10-fa69-4638-aac7-3cfc6b4e23be, rack_id: 27cfef72-d05b-490f-ac92-5c7cad830c2e, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64 } } } } target_sled = [fdb0:18c0:4d0d:9fb2::1]:12346
1452024-08-30T19:40:35.968ZINFOSledAgent (SledAgentSprocketsServer): Accepted connection file = sled-agent/src/bootstrap/sprockets_server.rs:66 remote_addr = [fdb0:18c0:4d0d:9fb2::1]:64573
1462024-08-30T19:40:35.968ZINFOSledAgent (SledAgentMain): Loading Sled Agent: StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: 289c9d10-fa69-4638-aac7-3cfc6b4e23be, rack_id: 27cfef72-d05b-490f-ac92-5c7cad830c2e, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64 } } } } file = sled-agent/src/bootstrap/server.rs:359
1472024-08-30T19:40:35.968ZINFOSledAgent (SledAgentMain): KeyManager: using hardcoded secret retriever file = sled-agent/src/bootstrap/server.rs:374
1482024-08-30T19:40:35.968ZINFOSledAgent (StorageManager): Received KeyManagerReady file = sled-storage/src/manager.rs:451
1492024-08-30T19:40:35.968ZINFOSledAgent (StorageManager): Loading storage config file = sled-storage/src/manager.rs:644
1502024-08-30T19:40:35.968ZINFOSledAgent: setting up sled agent server file = sled-agent/src/server.rs:44
1512024-08-30T19:40:35.968ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = internal-dns/src/resolver.rs:73
1522024-08-30T19:40:35.968ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:101::, width: 64 }
1532024-08-30T19:40:35.968ZINFOSledAgent (StorageManager): No ledger in /pool/int/fc9e5708-cd50-4950-8336-db011df04e25/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = load_disks_ledger
1542024-08-30T19:40:35.968ZINFOSledAgent (StorageManager): No ledger in /pool/int/17afe9d4-47cb-4799-8d3d-ad69badedc16/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = load_disks_ledger
1552024-08-30T19:40:35.968ZINFOSledAgent (StorageManager): No ledger of physical disks exists file = sled-storage/src/manager.rs:633
1562024-08-30T19:40:35.968ZINFOSledAgent (StorageManager): KeyManager ready, but no ledger detected file = sled-storage/src/manager.rs:661
1572024-08-30T19:40:35.968ZINFOSledAgent (StorageManager): automatically managing disks file = sled-storage/src/manager.rs:440
1582024-08-30T19:40:35.968ZINFOSledAgent (StorageResources): Synchronizing disk managment file = sled-storage/src/resources.rs:302
1592024-08-30T19:40:35.968ZINFOSledAgent (StorageManager): Successfully synchronized disks without error file = sled-storage/src/manager.rs:532 result = DisksManagementResult { status: [] }
1602024-08-30T19:40:35.970ZINFOSledAgent (ServiceManager): Enabling switch zone (new) file = sled-agent/src/services.rs:4163
1612024-08-30T19:40:35.970ZINFOSledAgent (ServiceManager): Starting switch zone file = sled-agent/src/services.rs:4545
1622024-08-30T19:40:36.003ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
1632024-08-30T19:40:36.025ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_switch file = illumos-utils/src/zone.rs:316
1642024-08-30T19:40:36.029ZINFOSledAgent: SledAgent::new(..) starting file = sled-agent/src/sled_agent.rs:388 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be
1652024-08-30T19:40:36.029ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
1662024-08-30T19:40:36.029ZINFOSledAgent: Requested swap device of size 64 GiB file = sled-agent/src/sled_agent.rs:400 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be
1672024-08-30T19:40:36.029ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1682024-08-30T19:40:36.032ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 76.402469ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "da8aaffa-0401-43db-a3cf-c668e6f7d617", "content-length": "124", "date": "Fri, 30 Aug 2024 19:40:35 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "da8aaffa-0401-43db-a3cf-c668e6f7d617" } file = clients/ddm-admin-client/src/lib.rs:177
1692024-08-30T19:40:36.042ZINFOSledAgent: attempting to create encrypted zvol: name="oxi_fc9e5708-cd50-4950-8336-db011df04e25/swap", size_gb=64 file = sled-agent/src/swap_device.rs:153
1702024-08-30T19:40:36.044ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_switch file = illumos-utils/src/zone.rs:354
1712024-08-30T19:40:36.066ZINFOSledAgent: successfully created encrypted zvol: name="oxi_fc9e5708-cd50-4950-8336-db011df04e25/swap", size_gb=64 file = sled-agent/src/swap_device.rs:249
1722024-08-30T19:40:36.066ZINFOSledAgent: adding swap device: swapname="/dev/zvol/dsk/oxi_fc9e5708-cd50-4950-8336-db011df04e25/swap" file = sled-agent/src/swap_device.rs:96
1732024-08-30T19:40:36.071ZINFOSledAgent: Mounting backing filesystems file = sled-agent/src/sled_agent.rs:415 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be
1742024-08-30T19:40:36.071ZINFOSledAgent (BackingFs): Processing fmd file = sled-agent/src/backing_fs.rs:127
1752024-08-30T19:40:36.071ZINFOSledAgent (BackingFs): Ensuring dataset oxi_fc9e5708-cd50-4950-8336-db011df04e25/backing/fmd file = sled-agent/src/backing_fs.rs:137
1762024-08-30T19:40:36.109ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1772024-08-30T19:40:36.110ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 141.782577ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "1d580b5b-8297-460c-a9d5-6df9475ae281", "content-length": "124", "date": "Fri, 30 Aug 2024 19:40:35 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "1d580b5b-8297-460c-a9d5-6df9475ae281" } file = clients/ddm-admin-client/src/lib.rs:177
1782024-08-30T19:40:36.156ZINFOSledAgent (BackingFs): Stopping service svc:/system/fmd:default file = sled-agent/src/backing_fs.rs:172
1792024-08-30T19:40:36.252ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1802024-08-30T19:40:36.253ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 86.383255ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "2555756a-59d8-45e7-bde3-278bf52e45a1", "content-length": "124", "date": "Fri, 30 Aug 2024 19:40:35 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "2555756a-59d8-45e7-bde3-278bf52e45a1" } file = clients/ddm-admin-client/src/lib.rs:177
1812024-08-30T19:40:36.340ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1822024-08-30T19:40:36.340ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 204.567721ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "6c8d3e15-dcec-4acd-8940-f3fb13a8237c", "content-length": "124", "date": "Fri, 30 Aug 2024 19:40:35 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "6c8d3e15-dcec-4acd-8940-f3fb13a8237c" } file = clients/ddm-admin-client/src/lib.rs:177
1832024-08-30T19:40:36.375ZINFOSledAgent (BackingFs): Mounting oxi_fc9e5708-cd50-4950-8336-db011df04e25/backing/fmd on /var/fm/fmd file = sled-agent/src/backing_fs.rs:180
1842024-08-30T19:40:36.387ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/rsrc file = sled-agent/src/backing_fs.rs:188
1852024-08-30T19:40:36.387ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/ckpt file = sled-agent/src/backing_fs.rs:188
1862024-08-30T19:40:36.387ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/xprt file = sled-agent/src/backing_fs.rs:188
1872024-08-30T19:40:36.387ZINFOSledAgent (BackingFs): Starting service svc:/system/fmd:default file = sled-agent/src/backing_fs.rs:194
1882024-08-30T19:40:36.547ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1892024-08-30T19:40:36.547ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 203.381297ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "1341adb3-1b59-41ed-a3b3-1aaeaae501af", "content-length": "124", "date": "Fri, 30 Aug 2024 19:40:35 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "1341adb3-1b59-41ed-a3b3-1aaeaae501af" } file = clients/ddm-admin-client/src/lib.rs:177
1902024-08-30T19:40:36.752ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1912024-08-30T19:40:36.753ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 251.605894ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "b6e68246-4ed4-40c3-b55c-b4e9c6d1a849", "content-length": "124", "date": "Fri, 30 Aug 2024 19:40:35 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "b6e68246-4ed4-40c3-b55c-b4e9c6d1a849" } file = clients/ddm-admin-client/src/lib.rs:177
1922024-08-30T19:40:37.005ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1932024-08-30T19:40:37.006ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 320.122055ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "ac6817db-313c-4be4-bd83-6c711d5febb8", "content-length": "124", "date": "Fri, 30 Aug 2024 19:40:37 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "ac6817db-313c-4be4-bd83-6c711d5febb8" } file = clients/ddm-admin-client/src/lib.rs:177
1942024-08-30T19:40:37.114ZINFOSledAgent: using '[AddrObject { interface: "net0", name: "ll" }, AddrObject { interface: "net1", name: "ll" }]' as data links for xde driver file = illumos-utils/src/opte/illumos.rs:90 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be
1952024-08-30T19:40:37.120ZINFOSledAgent (dropshot): listening file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:205 local_addr = [fd00:1122:3344:101::1]:48378 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be
1962024-08-30T19:40:37.120ZINFOSledAgent (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = internal-dns/src/resolver.rs:73 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be
1972024-08-30T19:40:37.120ZINFOSledAgent (producer-server): starting oximeter metric producer server address = [fd00:1122:3344:101::1]:48378 file = oximeter/producer/src/lib.rs:283 interval = 30s producer_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be
1982024-08-30T19:40:37.121ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 270.999383ms error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be
1992024-08-30T19:40:37.122ZINFOSledAgent (VmmReservoirManager): 50% of 102952026112 physical ram = 51476013056 bytes) file = sled-agent/src/vmm_reservoir.rs:267 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be
2002024-08-30T19:40:37.122ZINFOSledAgent (VmmReservoirManager): Setting reservoir size to 49090 MiB bytes file = sled-agent/src/vmm_reservoir.rs:275 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be
2012024-08-30T19:40:37.122ZINFOSledAgent (ServiceManager): sled agent started file = sled-agent/src/services.rs:914 underlay_address = fd00:1122:3344:101::1
2022024-08-30T19:40:37.122ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = internal-dns/src/resolver.rs:73
2032024-08-30T19:40:37.326ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
2042024-08-30T19:40:37.394ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 387.980006ms error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be
2052024-08-30T19:40:37.784ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 595.253011ms error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be
2062024-08-30T19:40:38.380ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 2.405547311s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be
2072024-08-30T19:40:39.571ZINFOSledAgent (ServiceManager): Setting up dendrite service file = sled-agent/src/services.rs:2639
2082024-08-30T19:40:39.571ZINFOSledAgent (ServiceManager): Setting up lldpd service file = sled-agent/src/services.rs:2857
2092024-08-30T19:40:39.571ZINFOSledAgent (ServiceManager): Setting up MGS service file = sled-agent/src/services.rs:2509
2102024-08-30T19:40:39.571ZINFOSledAgent (ServiceManager): Setting up wicketd service file = sled-agent/src/services.rs:2556
2112024-08-30T19:40:39.571ZINFOSledAgent (ServiceManager): Setting up mgd service file = sled-agent/src/services.rs:2938
2122024-08-30T19:40:39.571ZINFOSledAgent (ServiceManager): Setting up mg-ddm service file = sled-agent/src/services.rs:2980
2132024-08-30T19:40:39.573ZINFOSledAgent (ServiceManager): Setting up tfport service file = sled-agent/src/services.rs:2782
2142024-08-30T19:40:39.574ZINFOSledAgent (ServiceManager): Setting up Simulated SP service file = sled-agent/src/services.rs:2550
2152024-08-30T19:40:39.575ZINFOSledAgent (ServiceManager): Profile for oxz_switch: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService0'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/mgs"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="address" value='[::1]:12225'/> <propval type="astring" name="id" value='1a27df0f-5b46-452b-9341-ffa1d189a587'/> <propval type="astring" name="rack_id" value='27cfef72-d05b-490f-ac92-5c7cad830c2e'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/wicketd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="address" value='[::1]:12226'/> <propval type="astring" name="artifact-address" value='[fdb0:18c0:4d0d:9fb2::2]:12227'/> <propval type="astring" name="baseboard-file" value='/opt/oxide/baseboard.json'/> <propval type="astring" name="mgs-address" value='[::1]:12225'/> <propval type="astring" name="nexus-proxy-address" value='[::]:12229'/> <propval type="astring" name="rack-subnet" value='fd00:1122:3344::'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/switch_zone_setup"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="gz_local_link_addr" value='fe80::8:20ff:fe8c:7e5'/> <property type="astring" name="link_local_links"> <astring_list> <value_node value="tfportrear0_0" /> <value_node value="oxBootstrap0" /> </astring_list> </property> <propval type="astring" name="bootstrap_addr" value='fdb0:18c0:4d0d:9fb2::2'/> <propval type="astring" name="bootstrap_vnic" value='oxBootstrap0'/> <propval type="astring" name="baseboard_info" value='{"type":"pc","identifier":"buskin","model":"i86pc"}'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/dendrite"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="sled_id" value='289c9d10-fa69-4638-aac7-3cfc6b4e23be'/> <propval type="astring" name="rack_id" value='27cfef72-d05b-490f-ac92-5c7cad830c2e'/> <propval type="astring" name="address" value='[::1]:12224'/> <propval type="astring" name="front_ports" value='1'/> <propval type="astring" name="rear_ports" value='1'/> <propval type="astring" name="port_config" value='/opt/oxide/dendrite/misc/softnpu_single_sled_config.toml'/> <propval type="astring" name="mgmt" value='uds'/> <propval type="astring" name="uds_path" value='/opt/softnpu/stuff'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/tfport"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="host" value='[::1]'/> <propval type="astring" name="port" value='12224'/> <propval type="astring" name="flags" value='--sync-only'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/lldpd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="board_rev" value='softnpu_front_1_rear_1'/> <propval type="astring" name="scrimlet_id" value='buskin'/> <propval type="astring" name="scrimlet_model" value='i86pc'/> <propval type="astring" name="address" value='[::1]:12230'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/pumpkind"> </service> <service version="1" type="service" name="oxide/mgd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="sled_uuid" value='289c9d10-fa69-4638-aac7-3cfc6b4e23be'/> <propval type="astring" name="rack_uuid" value='27cfef72-d05b-490f-ac92-5c7cad830c2e'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/mg-ddm"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="mode" value='transit'/> <propval type="astring" name="dendrite" value='true'/> <propval type="astring" name="sled_uuid" value='289c9d10-fa69-4638-aac7-3cfc6b4e23be'/> <propval type="astring" name="rack_uuid" value='27cfef72-d05b-490f-ac92-5c7cad830c2e'/> <propval type="astring" name="interfaces" value='tfportrear0_0/ll'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/uplink"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
2162024-08-30T19:40:39.575ZINFOSledAgent (ServiceManager): Booting oxz_switch zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_switch
2172024-08-30T19:40:39.930ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 71.530945ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2182024-08-30T19:40:40.124ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 124.217647ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2192024-08-30T19:40:40.304ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 287.991896ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2202024-08-30T19:40:40.647ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 287.44676ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2212024-08-30T19:40:40.788ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 4.719190185s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be
2222024-08-30T19:40:41.028ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 1.065821861s file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2232024-08-30T19:40:42.122ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 828.303071ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2242024-08-30T19:40:43.105ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 1.262231448s file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2252024-08-30T19:40:44.528ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 500.801122ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2262024-08-30T19:40:45.063ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 613.169028ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2272024-08-30T19:40:45.509ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 6.328614404s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be
2282024-08-30T19:40:46.148ZINFOSledAgent (dropshot (SledAgent)): listening file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:205 local_addr = [fd00:1122:3344:101::1]:12345
2292024-08-30T19:40:46.148ZINFOSledAgent (SledAgentMain): Sled Agent loaded; recording configuration file = sled-agent/src/bootstrap/server.rs:429
2302024-08-30T19:40:46.148ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
2312024-08-30T19:40:46.148ZINFOSledAgent (SledAgentMain): Writing ledger to /pool/int/fc9e5708-cd50-4950-8336-db011df04e25/config/.sled-agent-request.json.tmp file = common/src/ledger.rs:199
2322024-08-30T19:40:46.148ZINFOSledAgent (SledAgentMain): Writing ledger to /pool/int/17afe9d4-47cb-4799-8d3d-ad69badedc16/config/.sled-agent-request.json.tmp file = common/src/ledger.rs:199
2332024-08-30T19:40:46.148ZINFOSledAgent (HardwareMonitor): Sled Agent Started file = sled-agent/src/hardware_monitor.rs:138
2342024-08-30T19:40:46.148ZINFOSledAgent (SledAgentSprocketsServer): Connection closed file = sled-agent/src/bootstrap/sprockets_server.rs:73 remote_addr = [fdb0:18c0:4d0d:9fb2::1]:64573
2352024-08-30T19:40:46.148ZINFOSledAgent (HardwareMonitor): Checking current full hardware snapshot disks = {} file = sled-agent/src/hardware_monitor.rs:271 underlay_network_info = Some((fd00:1122:3344:101::2, Some(RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56 }, infra_ip_first: 10.151.1.174, infra_ip_last: 10.151.1.174, ports: [PortConfigV2 { routes: [RouteConfig { destination: V4(Ipv4Net { addr: 0.0.0.0, width: 0 }), nexthop: 10.151.1.100, vlan_id: None, local_pref: None }], addresses: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: None, bgp_peers: [], autoneg: false, lldp: None }], bgp: [], bfd: [] })))
2362024-08-30T19:40:46.148ZINFOSledAgent (ServiceManager): Ensuring scrimlet services (enabling services) file = sled-agent/src/services.rs:3866
2372024-08-30T19:40:46.148ZINFOSledAgent (BootstrapAgentRssHandler): Peer agent initialized file = sled-agent/src/bootstrap/rss_handle.rs:108 peer = [fdb0:18c0:4d0d:9fb2::1]:12346
2382024-08-30T19:40:46.148ZINFOSledAgent (BootstrapAgentRssHandler): Initialized sled agent file = sled-agent/src/bootstrap/rss_handle.rs:232 target_sled = [fdb0:18c0:4d0d:9fb2::1]:12346
2392024-08-30T19:40:46.406ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
2402024-08-30T19:40:46.406ZINFOSledAgent (RSS): No ledger in /pool/int/fc9e5708-cd50-4950-8336-db011df04e25/config/rss-service-plan-v4.json file = common/src/ledger.rs:188
2412024-08-30T19:40:46.406ZINFOSledAgent (RSS): No ledger in /pool/int/17afe9d4-47cb-4799-8d3d-ad69badedc16/config/rss-service-plan-v4.json file = common/src/ledger.rs:188
2422024-08-30T19:40:46.406ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
2432024-08-30T19:40:46.406ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
2442024-08-30T19:40:46.406ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
2452024-08-30T19:40:46.411ZINFOSledAgent (ServiceManager): Re-enabling running switch zone (new address) file = sled-agent/src/services.rs:4187 new = [fd00:1122:3344:101::2, ::1] old = [::1]
2462024-08-30T19:40:46.411ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::2 exists file = sled-agent/src/services.rs:4203
2472024-08-30T19:40:46.411ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:101::2, prefix: 64 })) file = illumos-utils/src/running_zone.rs:534 zone = oxz_switch
2482024-08-30T19:40:46.672ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:49870
2492024-08-30T19:40:46.673ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
2502024-08-30T19:40:46.673ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 355 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:49870 req_id = a8425fc7-625c-495f-8184-d6b27ef495e3 response_code = 200 uri = /inventory
2512024-08-30T19:40:46.704ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:38948
2522024-08-30T19:40:46.704ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 97 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:38948 req_id = e736a0ac-404b-4588-90ab-678306ce4120 response_code = 200 uri = /sled-role
2532024-08-30T19:40:46.705ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
2542024-08-30T19:40:46.705ZINFOSledAgent (RSS): Writing ledger to /pool/int/fc9e5708-cd50-4950-8336-db011df04e25/config/.rss-service-plan-v4.json.tmp file = common/src/ledger.rs:199
2552024-08-30T19:40:46.705ZINFOSledAgent (RSS): Writing ledger to /pool/int/17afe9d4-47cb-4799-8d3d-ad69badedc16/config/.rss-service-plan-v4.json.tmp file = common/src/ledger.rs:199
2562024-08-30T19:40:46.706ZINFOSledAgent (RSS): Service plan written to storage file = sled-agent/src/rack_setup/plan/service.rs:1001
2572024-08-30T19:40:46.736ZINFOSledAgent (RSS): attempting to set up sled's storage: OmicronPhysicalDisksConfig { generation: Generation(1), disks: [OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, id: af492c8c-1b1e-48c6-b05f-50c3da4f8e5d, pool_id: 83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, id: a28d7bf2-5d6d-47bd-8aca-b32c37e0bc34, pool_id: 1693cbb3-d027-4679-ba05-f26fe21ef04a (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, id: 9ec8fb89-5b41-4f65-b7ae-eb5773de2d97, pool_id: 161cdb67-d423-452c-be58-7bca2891066e (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, id: c3eec841-7e72-4a0f-87fc-c4a8d8c12cca, pool_id: 3fd4f884-7e56-4c58-871f-b010d32df20b (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, id: e3fa4c24-c427-4373-ac00-421649f0d118, pool_id: 6113cfc6-d07f-470a-963b-4465a62b65f1 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, id: 7b1aabde-2fa3-4016-9b10-e944e08c682b, pool_id: 9a6b9e1b-b715-4f19-b00f-4a1c2b109fdc (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, id: 26923fc4-08cd-4e1d-9fbc-ddabee89b218, pool_id: 50a6c5cd-19db-4cbc-bda0-e365b45637b0 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, id: d1517a05-5d50-4dfe-b55d-fb4c1fccf7a2, pool_id: a2ff29f8-f12c-4d2f-add0-1d5faf6635ee (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, id: f35bfd31-efab-428d-95b6-9d1b677e641c, pool_id: d69ccb3d-8bd7-402d-b7bd-712c30af96dc (zpool) }] } file = sled-agent/src/rack_setup/service.rs:355 sled_address = [fd00:1122:3344:101::1]:12345
2582024-08-30T19:40:46.737ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:60168
2592024-08-30T19:40:46.737ZINFOSledAgent: physical disks ensure file = sled-agent/src/sled_agent.rs:856 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be
2602024-08-30T19:40:46.737ZINFOSledAgent (StorageManager): Received OmicronPhysicalDisksEnsure { config: OmicronPhysicalDisksConfig { generation: Generation(1), disks: [OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, id: af492c8c-1b1e-48c6-b05f-50c3da4f8e5d, pool_id: 83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, id: a28d7bf2-5d6d-47bd-8aca-b32c37e0bc34, pool_id: 1693cbb3-d027-4679-ba05-f26fe21ef04a (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, id: 9ec8fb89-5b41-4f65-b7ae-eb5773de2d97, pool_id: 161cdb67-d423-452c-be58-7bca2891066e (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, id: c3eec841-7e72-4a0f-87fc-c4a8d8c12cca, pool_id: 3fd4f884-7e56-4c58-871f-b010d32df20b (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, id: e3fa4c24-c427-4373-ac00-421649f0d118, pool_id: 6113cfc6-d07f-470a-963b-4465a62b65f1 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, id: 7b1aabde-2fa3-4016-9b10-e944e08c682b, pool_id: 9a6b9e1b-b715-4f19-b00f-4a1c2b109fdc (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, id: 26923fc4-08cd-4e1d-9fbc-ddabee89b218, pool_id: 50a6c5cd-19db-4cbc-bda0-e365b45637b0 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, id: d1517a05-5d50-4dfe-b55d-fb4c1fccf7a2, pool_id: a2ff29f8-f12c-4d2f-add0-1d5faf6635ee (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, id: f35bfd31-efab-428d-95b6-9d1b677e641c, pool_id: d69ccb3d-8bd7-402d-b7bd-712c30af96dc (zpool) }] }, tx: ... } file = sled-storage/src/manager.rs:451
2612024-08-30T19:40:46.737ZINFOSledAgent (StorageManager): No ledger in /pool/int/fc9e5708-cd50-4950-8336-db011df04e25/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = omicron_physical_disks_ensure
2622024-08-30T19:40:46.737ZINFOSledAgent (StorageManager): No ledger in /pool/int/17afe9d4-47cb-4799-8d3d-ad69badedc16/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = omicron_physical_disks_ensure
2632024-08-30T19:40:46.737ZINFOSledAgent (StorageManager): No previously-stored 'requested disks', creating new ledger file = sled-storage/src/manager.rs:866 request = omicron_physical_disks_ensure
2642024-08-30T19:40:46.737ZINFOSledAgent (StorageResources): Synchronizing disk managment file = sled-storage/src/resources.rs:302
2652024-08-30T19:40:46.737ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:342
2662024-08-30T19:40:46.737ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
2672024-08-30T19:40:46.737ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } path = /scratch/u2_0.vdev
2682024-08-30T19:40:46.888ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_0.vdev file = sled-hardware/src/disk.rs:360
2692024-08-30T19:40:46.888ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = 83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 (zpool)
2702024-08-30T19:40:47.149ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/289c9d10-fa69-4638-aac7-3cfc6b4e23be): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/probe_manager.rs:230 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be
2712024-08-30T19:40:47.347ZINFOSledAgent (VmmReservoirManager): Finished setting reservoir size to 49090 MiB bytes file = sled-agent/src/vmm_reservoir.rs:279 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be
2722024-08-30T19:40:47.411ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:174 zpool = ZpoolName { id: 83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 (zpool), kind: External }
2732024-08-30T19:40:47.420ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:218
2742024-08-30T19:40:47.420ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:220
2752024-08-30T19:40:47.420ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:224
2762024-08-30T19:40:47.420ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:226
2772024-08-30T19:40:47.420ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key")
2782024-08-30T19:40:47.420ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key")
2792024-08-30T19:40:47.420ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:238
2802024-08-30T19:40:47.455ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
2812024-08-30T19:40:47.464ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9/crypt/zone file = sled-storage/src/dataset.rs:281
2822024-08-30T19:40:47.640ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 file = sled-storage/src/dataset.rs:367
2832024-08-30T19:40:47.648ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:440
2842024-08-30T19:40:47.648ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:357
2852024-08-30T19:40:47.648ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:342
2862024-08-30T19:40:47.648ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
2872024-08-30T19:40:47.648ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } path = /scratch/u2_1.vdev
2882024-08-30T19:40:47.658ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_1.vdev file = sled-hardware/src/disk.rs:360
2892024-08-30T19:40:47.658ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = 1693cbb3-d027-4679-ba05-f26fe21ef04a (zpool)
2902024-08-30T19:40:47.749ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:174 zpool = ZpoolName { id: 1693cbb3-d027-4679-ba05-f26fe21ef04a (zpool), kind: External }
2912024-08-30T19:40:47.758ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:218
2922024-08-30T19:40:47.758ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:220
2932024-08-30T19:40:47.758ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:224
2942024-08-30T19:40:47.758ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:226
2952024-08-30T19:40:47.758ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key")
2962024-08-30T19:40:47.758ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key")
2972024-08-30T19:40:47.758ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:238
2982024-08-30T19:40:47.792ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
2992024-08-30T19:40:47.800ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_1693cbb3-d027-4679-ba05-f26fe21ef04a/crypt/zone file = sled-storage/src/dataset.rs:281
3002024-08-30T19:40:47.978ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_1693cbb3-d027-4679-ba05-f26fe21ef04a file = sled-storage/src/dataset.rs:367
3012024-08-30T19:40:47.986ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:440
3022024-08-30T19:40:47.986ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:357
3032024-08-30T19:40:47.986ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:342
3042024-08-30T19:40:47.986ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3052024-08-30T19:40:47.986ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } path = /scratch/u2_2.vdev
3062024-08-30T19:40:47.996ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_2.vdev file = sled-hardware/src/disk.rs:360
3072024-08-30T19:40:47.996ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = 161cdb67-d423-452c-be58-7bca2891066e (zpool)
3082024-08-30T19:40:48.091ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/dataset.rs:174 zpool = ZpoolName { id: 161cdb67-d423-452c-be58-7bca2891066e (zpool), kind: External }
3092024-08-30T19:40:48.099ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/dataset.rs:218
3102024-08-30T19:40:48.099ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:220
3112024-08-30T19:40:48.099ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:224
3122024-08-30T19:40:48.099ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:226
3132024-08-30T19:40:48.099ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key")
3142024-08-30T19:40:48.099ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key")
3152024-08-30T19:40:48.099ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:238
3162024-08-30T19:40:48.134ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3172024-08-30T19:40:48.142ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_161cdb67-d423-452c-be58-7bca2891066e/crypt/zone file = sled-storage/src/dataset.rs:281
3182024-08-30T19:40:48.154ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/289c9d10-fa69-4638-aac7-3cfc6b4e23be): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/probe_manager.rs:230 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be
3192024-08-30T19:40:48.316ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_161cdb67-d423-452c-be58-7bca2891066e file = sled-storage/src/dataset.rs:367
3202024-08-30T19:40:48.324ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:440
3212024-08-30T19:40:48.324ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:357
3222024-08-30T19:40:48.324ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:342
3232024-08-30T19:40:48.324ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3242024-08-30T19:40:48.324ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } path = /scratch/u2_3.vdev
3252024-08-30T19:40:48.334ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_3.vdev file = sled-hardware/src/disk.rs:360
3262024-08-30T19:40:48.334ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = 3fd4f884-7e56-4c58-871f-b010d32df20b (zpool)
3272024-08-30T19:40:48.429ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/dataset.rs:174 zpool = ZpoolName { id: 3fd4f884-7e56-4c58-871f-b010d32df20b (zpool), kind: External }
3282024-08-30T19:40:48.437ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/dataset.rs:218
3292024-08-30T19:40:48.437ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:220
3302024-08-30T19:40:48.437ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:224
3312024-08-30T19:40:48.437ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:226
3322024-08-30T19:40:48.437ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key")
3332024-08-30T19:40:48.437ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key")
3342024-08-30T19:40:48.437ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:238
3352024-08-30T19:40:48.473ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3362024-08-30T19:40:48.481ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_3fd4f884-7e56-4c58-871f-b010d32df20b/crypt/zone file = sled-storage/src/dataset.rs:281
3372024-08-30T19:40:48.662ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_3fd4f884-7e56-4c58-871f-b010d32df20b file = sled-storage/src/dataset.rs:367
3382024-08-30T19:40:48.670ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:440
3392024-08-30T19:40:48.670ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:357
3402024-08-30T19:40:48.670ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:342
3412024-08-30T19:40:48.670ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3422024-08-30T19:40:48.670ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } path = /scratch/u2_4.vdev
3432024-08-30T19:40:48.680ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_4.vdev file = sled-hardware/src/disk.rs:360
3442024-08-30T19:40:48.680ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = 6113cfc6-d07f-470a-963b-4465a62b65f1 (zpool)
3452024-08-30T19:40:48.780ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/dataset.rs:174 zpool = ZpoolName { id: 6113cfc6-d07f-470a-963b-4465a62b65f1 (zpool), kind: External }
3462024-08-30T19:40:48.788ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/dataset.rs:218
3472024-08-30T19:40:48.788ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:220
3482024-08-30T19:40:48.788ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:224
3492024-08-30T19:40:48.788ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:226
3502024-08-30T19:40:48.788ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key")
3512024-08-30T19:40:48.789ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key")
3522024-08-30T19:40:48.789ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:238
3532024-08-30T19:40:48.825ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3542024-08-30T19:40:48.833ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_6113cfc6-d07f-470a-963b-4465a62b65f1/crypt/zone file = sled-storage/src/dataset.rs:281
3552024-08-30T19:40:49.009ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_6113cfc6-d07f-470a-963b-4465a62b65f1 file = sled-storage/src/dataset.rs:367
3562024-08-30T19:40:49.017ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:440
3572024-08-30T19:40:49.018ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:357
3582024-08-30T19:40:49.018ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:342
3592024-08-30T19:40:49.018ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3602024-08-30T19:40:49.018ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } path = /scratch/u2_5.vdev
3612024-08-30T19:40:49.027ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_5.vdev file = sled-hardware/src/disk.rs:360
3622024-08-30T19:40:49.027ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = 9a6b9e1b-b715-4f19-b00f-4a1c2b109fdc (zpool)
3632024-08-30T19:40:49.123ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/dataset.rs:174 zpool = ZpoolName { id: 9a6b9e1b-b715-4f19-b00f-4a1c2b109fdc (zpool), kind: External }
3642024-08-30T19:40:49.132ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/dataset.rs:218
3652024-08-30T19:40:49.132ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:220
3662024-08-30T19:40:49.132ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:224
3672024-08-30T19:40:49.132ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:226
3682024-08-30T19:40:49.132ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key")
3692024-08-30T19:40:49.132ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key")
3702024-08-30T19:40:49.132ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:238
3712024-08-30T19:40:49.155ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/289c9d10-fa69-4638-aac7-3cfc6b4e23be): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/probe_manager.rs:230 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be
3722024-08-30T19:40:49.169ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3732024-08-30T19:40:49.177ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_9a6b9e1b-b715-4f19-b00f-4a1c2b109fdc/crypt/zone file = sled-storage/src/dataset.rs:281
3742024-08-30T19:40:49.346ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_9a6b9e1b-b715-4f19-b00f-4a1c2b109fdc file = sled-storage/src/dataset.rs:367
3752024-08-30T19:40:49.353ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:440
3762024-08-30T19:40:49.353ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:357
3772024-08-30T19:40:49.353ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:342
3782024-08-30T19:40:49.353ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3792024-08-30T19:40:49.353ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } path = /scratch/u2_6.vdev
3802024-08-30T19:40:49.363ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_6.vdev file = sled-hardware/src/disk.rs:360
3812024-08-30T19:40:49.363ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = 50a6c5cd-19db-4cbc-bda0-e365b45637b0 (zpool)
3822024-08-30T19:40:49.463ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/dataset.rs:174 zpool = ZpoolName { id: 50a6c5cd-19db-4cbc-bda0-e365b45637b0 (zpool), kind: External }
3832024-08-30T19:40:49.471ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/dataset.rs:218
3842024-08-30T19:40:49.472ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:220
3852024-08-30T19:40:49.472ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:224
3862024-08-30T19:40:49.472ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:226
3872024-08-30T19:40:49.472ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key")
3882024-08-30T19:40:49.472ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key")
3892024-08-30T19:40:49.472ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:238
3902024-08-30T19:40:49.509ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3912024-08-30T19:40:49.516ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_50a6c5cd-19db-4cbc-bda0-e365b45637b0/crypt/zone file = sled-storage/src/dataset.rs:281
3922024-08-30T19:40:49.691ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_50a6c5cd-19db-4cbc-bda0-e365b45637b0 file = sled-storage/src/dataset.rs:367
3932024-08-30T19:40:49.699ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:440
3942024-08-30T19:40:49.699ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:357
3952024-08-30T19:40:49.699ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:342
3962024-08-30T19:40:49.699ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3972024-08-30T19:40:49.699ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } path = /scratch/u2_7.vdev
3982024-08-30T19:40:49.708ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_7.vdev file = sled-hardware/src/disk.rs:360
3992024-08-30T19:40:49.708ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = a2ff29f8-f12c-4d2f-add0-1d5faf6635ee (zpool)
4002024-08-30T19:40:49.805ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/dataset.rs:174 zpool = ZpoolName { id: a2ff29f8-f12c-4d2f-add0-1d5faf6635ee (zpool), kind: External }
4012024-08-30T19:40:49.813ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/dataset.rs:218
4022024-08-30T19:40:49.813ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:220
4032024-08-30T19:40:49.813ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:224
4042024-08-30T19:40:49.813ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:226
4052024-08-30T19:40:49.813ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key")
4062024-08-30T19:40:49.813ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key")
4072024-08-30T19:40:49.813ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:238
4082024-08-30T19:40:49.849ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
4092024-08-30T19:40:49.857ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_a2ff29f8-f12c-4d2f-add0-1d5faf6635ee/crypt/zone file = sled-storage/src/dataset.rs:281
4102024-08-30T19:40:50.033ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_a2ff29f8-f12c-4d2f-add0-1d5faf6635ee file = sled-storage/src/dataset.rs:367
4112024-08-30T19:40:50.040ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:440
4122024-08-30T19:40:50.040ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:357
4132024-08-30T19:40:50.040ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:342
4142024-08-30T19:40:50.040ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
4152024-08-30T19:40:50.040ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } path = /scratch/u2_8.vdev
4162024-08-30T19:40:50.049ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_8.vdev file = sled-hardware/src/disk.rs:360
4172024-08-30T19:40:50.049ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = d69ccb3d-8bd7-402d-b7bd-712c30af96dc (zpool)
4182024-08-30T19:40:50.149ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/dataset.rs:174 zpool = ZpoolName { id: d69ccb3d-8bd7-402d-b7bd-712c30af96dc (zpool), kind: External }
4192024-08-30T19:40:50.157ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/dataset.rs:218
4202024-08-30T19:40:50.157ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:220
4212024-08-30T19:40:50.157ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:224
4222024-08-30T19:40:50.157ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:226
4232024-08-30T19:40:50.157ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key")
4242024-08-30T19:40:50.157ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key")
4252024-08-30T19:40:50.157ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:238
4262024-08-30T19:40:50.162ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/289c9d10-fa69-4638-aac7-3cfc6b4e23be): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/probe_manager.rs:230 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be
4272024-08-30T19:40:50.196ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
4282024-08-30T19:40:50.204ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_d69ccb3d-8bd7-402d-b7bd-712c30af96dc/crypt/zone file = sled-storage/src/dataset.rs:281
4292024-08-30T19:40:50.382ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_d69ccb3d-8bd7-402d-b7bd-712c30af96dc file = sled-storage/src/dataset.rs:367
4302024-08-30T19:40:50.390ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:440
4312024-08-30T19:40:50.390ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:357
4322024-08-30T19:40:50.390ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/fc9e5708-cd50-4950-8336-db011df04e25/config/.omicron-physical-disks.json.tmp file = common/src/ledger.rs:199 request = omicron_physical_disks_ensure
4332024-08-30T19:40:50.390ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: fc9e5708-cd50-4950-8336-db011df04e25 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 17afe9d4-47cb-4799-8d3d-ad69badedc16 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 1693cbb3-d027-4679-ba05-f26fe21ef04a (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 161cdb67-d423-452c-be58-7bca2891066e (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 3fd4f884-7e56-4c58-871f-b010d32df20b (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 6113cfc6-d07f-470a-963b-4465a62b65f1 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 9a6b9e1b-b715-4f19-b00f-4a1c2b109fdc (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 50a6c5cd-19db-4cbc-bda0-e365b45637b0 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: a2ff29f8-f12c-4d2f-add0-1d5faf6635ee (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: d69ccb3d-8bd7-402d-b7bd-712c30af96dc (zpool), kind: External } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
4342024-08-30T19:40:50.390ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
4352024-08-30T19:40:50.390ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
4362024-08-30T19:40:50.390ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/17afe9d4-47cb-4799-8d3d-ad69badedc16/config/.omicron-physical-disks.json.tmp file = common/src/ledger.rs:199 request = omicron_physical_disks_ensure
4372024-08-30T19:40:50.391ZINFOSledAgent: physical disks ensure: Updated storage file = sled-agent/src/sled_agent.rs:860 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be
4382024-08-30T19:40:50.391ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
4392024-08-30T19:40:50.391ZINFOSledAgent: physical disks ensure: Propagating new generation of disks file = sled-agent/src/sled_agent.rs:886 generation = Generation(1) sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be
4402024-08-30T19:40:50.391ZINFOSledAgent: physical disks ensure: Updated storage monitor file = sled-agent/src/sled_agent.rs:891 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be
4412024-08-30T19:40:50.391ZINFOSledAgent: physical disks ensure: Updated zone bundler file = sled-agent/src/sled_agent.rs:896 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be
4422024-08-30T19:40:50.391ZINFOSledAgent (ProbeManager): use_only_these_disks: Processing new request file = sled-agent/src/probe_manager.rs:121 gen = Generation(1) sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be
4432024-08-30T19:40:50.391ZINFOSledAgent: physical disks ensure: Updated probes file = sled-agent/src/sled_agent.rs:902 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be
4442024-08-30T19:40:50.391ZINFOSledAgent (InstanceManager): use_only_these_disks: Processing new request file = sled-agent/src/instance_manager.rs:803 gen = Generation(1)
4452024-08-30T19:40:50.391ZINFOSledAgent: physical disks ensure: Updated instances file = sled-agent/src/sled_agent.rs:907 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be
4462024-08-30T19:40:50.391ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3654018 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:60168 req_id = ef83d5e0-6e3d-49f5-84ae-879890b4c45f response_code = 200 uri = /omicron-physical-disks
4472024-08-30T19:40:50.423ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(2), zones: [OmicronZoneConfig { id: 8b642bba-10a6-428b-9af7-48d5f8622e03, underlay_address: fd00:1122:3344:1::1, filesystem_pool: Some(ZpoolName { id: 83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: c4910af2-5760-4c5f-bb62-3b2357eab858, underlay_address: fd00:1122:3344:2::1, filesystem_pool: Some(ZpoolName { id: 1693cbb3-d027-4679-ba05-f26fe21ef04a (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 1693cbb3-d027-4679-ba05-f26fe21ef04a (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: b5d7b957-0082-4dc3-b742-d6e7150a43c2, underlay_address: fd00:1122:3344:3::1, filesystem_pool: Some(ZpoolName { id: 161cdb67-d423-452c-be58-7bca2891066e (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 161cdb67-d423-452c-be58-7bca2891066e (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }] } file = sled-agent/src/rack_setup/service.rs:438 sled_address = [fd00:1122:3344:101::1]:12345
4482024-08-30T19:40:50.423ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:60102
4492024-08-30T19:40:50.423ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 8b642bba-10a6-428b-9af7-48d5f8622e03, dataset_name: DatasetName { pool_name: ZpoolName { id: 83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
4502024-08-30T19:40:50.423ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 8b642bba-10a6-428b-9af7-48d5f8622e03, dataset_name: DatasetName { pool_name: ZpoolName { id: 83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 (zpool), kind: External }, kind: InternalDns }, responder: ... }
4512024-08-30T19:40:50.476ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c4910af2-5760-4c5f-bb62-3b2357eab858, dataset_name: DatasetName { pool_name: ZpoolName { id: 1693cbb3-d027-4679-ba05-f26fe21ef04a (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
4522024-08-30T19:40:50.476ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: c4910af2-5760-4c5f-bb62-3b2357eab858, dataset_name: DatasetName { pool_name: ZpoolName { id: 1693cbb3-d027-4679-ba05-f26fe21ef04a (zpool), kind: External }, kind: InternalDns }, responder: ... }
4532024-08-30T19:40:50.527ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: b5d7b957-0082-4dc3-b742-d6e7150a43c2, dataset_name: DatasetName { pool_name: ZpoolName { id: 161cdb67-d423-452c-be58-7bca2891066e (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
4542024-08-30T19:40:50.527ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: b5d7b957-0082-4dc3-b742-d6e7150a43c2, dataset_name: DatasetName { pool_name: ZpoolName { id: 161cdb67-d423-452c-be58-7bca2891066e (zpool), kind: External }, kind: InternalDns }, responder: ... }
4552024-08-30T19:40:50.576ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
4562024-08-30T19:40:50.577ZINFOSledAgent (ServiceManager): No ledger in /pool/int/fc9e5708-cd50-4950-8336-db011df04e25/config/omicron-zones.json file = common/src/ledger.rs:188
4572024-08-30T19:40:50.577ZINFOSledAgent (ServiceManager): No ledger in /pool/int/17afe9d4-47cb-4799-8d3d-ad69badedc16/config/omicron-zones.json file = common/src/ledger.rs:188
4582024-08-30T19:40:50.577ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
4592024-08-30T19:40:50.621ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
4602024-08-30T19:40:50.621ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
4612024-08-30T19:40:50.621ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
4622024-08-30T19:40:50.672ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_b5d7b957-0082-4dc3-b742-d6e7150a43c2 file = illumos-utils/src/zone.rs:316
4632024-08-30T19:40:50.676ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_8b642bba-10a6-428b-9af7-48d5f8622e03 file = illumos-utils/src/zone.rs:316
4642024-08-30T19:40:50.680ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_c4910af2-5760-4c5f-bb62-3b2357eab858 file = illumos-utils/src/zone.rs:316
4652024-08-30T19:40:50.691ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_b5d7b957-0082-4dc3-b742-d6e7150a43c2 file = illumos-utils/src/zone.rs:354
4662024-08-30T19:40:50.696ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_8b642bba-10a6-428b-9af7-48d5f8622e03 file = illumos-utils/src/zone.rs:354
4672024-08-30T19:40:50.699ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_c4910af2-5760-4c5f-bb62-3b2357eab858 file = illumos-utils/src/zone.rs:354
4682024-08-30T19:40:51.164ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/289c9d10-fa69-4638-aac7-3cfc6b4e23be): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/probe_manager.rs:230 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be
4692024-08-30T19:40:52.169ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_b5d7b957-0082-4dc3-b742-d6e7150a43c2: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService1'/> <propval type="astring" name="gateway" value='fd00:1122:3344:3::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:3::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:3::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:3::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
4702024-08-30T19:40:52.169ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:3::, width: 64 }
4712024-08-30T19:40:52.170ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_b5d7b957-0082-4dc3-b742-d6e7150a43c2 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_b5d7b957-0082-4dc3-b742-d6e7150a43c2
4722024-08-30T19:40:52.653ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_8b642bba-10a6-428b-9af7-48d5f8622e03: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService2'/> <propval type="astring" name="gateway" value='fd00:1122:3344:1::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:1::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:1::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:1::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
4732024-08-30T19:40:53.131ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:1::, width: 64 }
4742024-08-30T19:40:53.131ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_c4910af2-5760-4c5f-bb62-3b2357eab858: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService3'/> <propval type="astring" name="gateway" value='fd00:1122:3344:2::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:2::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:2::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:2::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
4752024-08-30T19:40:53.131ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:2::, width: 64 }
4762024-08-30T19:40:53.139ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_b5d7b957-0082-4dc3-b742-d6e7150a43c2") failed: Property not found. retry in 36.160802ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_b5d7b957-0082-4dc3-b742-d6e7150a43c2
4772024-08-30T19:40:53.139ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_8b642bba-10a6-428b-9af7-48d5f8622e03 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_8b642bba-10a6-428b-9af7-48d5f8622e03
4782024-08-30T19:40:53.143ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_c4910af2-5760-4c5f-bb62-3b2357eab858 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_c4910af2-5760-4c5f-bb62-3b2357eab858
4792024-08-30T19:40:53.184ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_b5d7b957-0082-4dc3-b742-d6e7150a43c2") failed: Property not found. retry in 67.141837ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_b5d7b957-0082-4dc3-b742-d6e7150a43c2
4802024-08-30T19:40:53.261ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_b5d7b957-0082-4dc3-b742-d6e7150a43c2") failed: Property not found. retry in 254.163013ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_b5d7b957-0082-4dc3-b742-d6e7150a43c2
4812024-08-30T19:40:53.269ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c4910af2-5760-4c5f-bb62-3b2357eab858") failed: Property not found. retry in 61.104045ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c4910af2-5760-4c5f-bb62-3b2357eab858
4822024-08-30T19:40:53.275ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_8b642bba-10a6-428b-9af7-48d5f8622e03") failed: Property not found. retry in 33.684526ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_8b642bba-10a6-428b-9af7-48d5f8622e03
4832024-08-30T19:40:53.316ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_8b642bba-10a6-428b-9af7-48d5f8622e03") failed: Property not found. retry in 135.462657ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_8b642bba-10a6-428b-9af7-48d5f8622e03
4842024-08-30T19:40:53.410ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c4910af2-5760-4c5f-bb62-3b2357eab858") failed: Property not found. retry in 117.873856ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c4910af2-5760-4c5f-bb62-3b2357eab858
4852024-08-30T19:40:53.467ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_8b642bba-10a6-428b-9af7-48d5f8622e03") failed: Property not found. retry in 174.412612ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_8b642bba-10a6-428b-9af7-48d5f8622e03
4862024-08-30T19:40:53.552ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_b5d7b957-0082-4dc3-b742-d6e7150a43c2") failed: Property not found. retry in 488.993048ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_b5d7b957-0082-4dc3-b742-d6e7150a43c2
4872024-08-30T19:40:53.581ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c4910af2-5760-4c5f-bb62-3b2357eab858") failed: Property not found. retry in 268.948063ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c4910af2-5760-4c5f-bb62-3b2357eab858
4882024-08-30T19:40:53.716ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_8b642bba-10a6-428b-9af7-48d5f8622e03") failed: Property not found. retry in 430.898528ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_8b642bba-10a6-428b-9af7-48d5f8622e03
4892024-08-30T19:40:53.859ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c4910af2-5760-4c5f-bb62-3b2357eab858") failed: Property not found. retry in 576.442237ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c4910af2-5760-4c5f-bb62-3b2357eab858
4902024-08-30T19:40:54.050ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_b5d7b957-0082-4dc3-b742-d6e7150a43c2") failed: Property not found. retry in 1.184294739s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_b5d7b957-0082-4dc3-b742-d6e7150a43c2
4912024-08-30T19:40:54.157ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_8b642bba-10a6-428b-9af7-48d5f8622e03") failed: Property not found. retry in 748.458725ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_8b642bba-10a6-428b-9af7-48d5f8622e03
4922024-08-30T19:40:54.447ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c4910af2-5760-4c5f-bb62-3b2357eab858") failed: Property not found. retry in 760.860339ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c4910af2-5760-4c5f-bb62-3b2357eab858
4932024-08-30T19:40:54.919ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_8b642bba-10a6-428b-9af7-48d5f8622e03") failed: Property not found. retry in 1.286844041s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_8b642bba-10a6-428b-9af7-48d5f8622e03
4942024-08-30T19:40:55.217ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c4910af2-5760-4c5f-bb62-3b2357eab858") failed: Property not found. retry in 1.04946857s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c4910af2-5760-4c5f-bb62-3b2357eab858
4952024-08-30T19:40:55.243ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_b5d7b957-0082-4dc3-b742-d6e7150a43c2") failed: Property not found. retry in 1.481103023s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_b5d7b957-0082-4dc3-b742-d6e7150a43c2
4962024-08-30T19:40:55.957ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::2 exists - OK file = sled-agent/src/services.rs:4211
4972024-08-30T19:40:55.957ZINFOSledAgent (ServiceManager): Ensuring there is a default route file = sled-agent/src/services.rs:4221 gateway = fd00:1122:3344:101::1
4982024-08-30T19:40:55.963ZINFOSledAgent (ServiceManager): configuring dendrite service file = sled-agent/src/services.rs:4300
4992024-08-30T19:40:56.093ZINFOSledAgent (ServiceManager): refreshed dendrite service with new configuration file = sled-agent/src/services.rs:4348
5002024-08-30T19:40:56.093ZINFOSledAgent (ServiceManager): configuring lldp service file = sled-agent/src/services.rs:4375
5012024-08-30T19:40:56.140ZINFOSledAgent (ServiceManager): refreshed lldpd service with new configuration file = sled-agent/src/services.rs:4388
5022024-08-30T19:40:56.140ZINFOSledAgent (ServiceManager): configuring MGS service file = sled-agent/src/services.rs:4252
5032024-08-30T19:40:56.195ZINFOSledAgent (ServiceManager): refreshed MGS service with new configuration file = sled-agent/src/services.rs:4294
5042024-08-30T19:40:56.195ZINFOSledAgent (ServiceManager): configuring wicketd file = sled-agent/src/services.rs:4355 rack_subnet = fd00:1122:3344::
5052024-08-30T19:40:56.217ZINFOSledAgent (ServiceManager): refreshed wicketd service with new configuration file = sled-agent/src/services.rs:4366
5062024-08-30T19:40:56.217ZINFOSledAgent (ServiceManager): configuring mgd service file = sled-agent/src/services.rs:4408
5072024-08-30T19:40:56.218ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_8b642bba-10a6-428b-9af7-48d5f8622e03") failed: Property not found. retry in 1.158876018s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_8b642bba-10a6-428b-9af7-48d5f8622e03
5082024-08-30T19:40:56.251ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::2]:43647
5092024-08-30T19:40:56.252ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 75 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::2]:43647 req_id = e40d2d74-cc96-4b96-81a3-251956bc9614 response_code = 200 uri = /sled-identifiers
5102024-08-30T19:40:56.275ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c4910af2-5760-4c5f-bb62-3b2357eab858") failed: Property not found. retry in 1.414609831s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c4910af2-5760-4c5f-bb62-3b2357eab858
5112024-08-30T19:40:56.311ZINFOSledAgent (ServiceManager): refreshed mgd service with new configuration file = sled-agent/src/services.rs:4440
5122024-08-30T19:40:56.311ZINFOSledAgent (ServiceManager): configuring mg-ddm service file = sled-agent/src/services.rs:4446
5132024-08-30T19:40:56.420ZINFOSledAgent (ServiceManager): refreshed mg-ddm service with new configuration file = sled-agent/src/services.rs:4487
5142024-08-30T19:40:56.420ZINFOSledAgent (ServiceManager): Determining physical location of our switch zone at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:366
5152024-08-30T19:40:56.452ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/289c9d10-fa69-4638-aac7-3cfc6b4e23be): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/nexus.rs:571 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be
5162024-08-30T19:40:56.453ZINFOSledAgent (ServiceManager): Initializing 1 Uplinks on Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:414
5172024-08-30T19:40:56.485ZINFOSledAgent (ServiceManager): Building Port Configuration file = sled-agent/src/bootstrap/early_networking.rs:676
5182024-08-30T19:40:56.485ZINFOSledAgent (ServiceManager): Checking dendrite uptime file = sled-agent/src/bootstrap/early_networking.rs:714
5192024-08-30T19:40:56.486ZINFOSledAgent (ServiceManager): Dendrite online file = sled-agent/src/bootstrap/early_networking.rs:717 uptime = 0
5202024-08-30T19:40:56.486ZINFOSledAgent (ServiceManager): Configuring default uplink on switch config = PortSettings {\n links: {\n "0": LinkSettings {\n addrs: [\n 10.151.1.174,\n ],\n params: LinkCreate {\n autoneg: false,\n fec: None,\n kr: false,\n lane: Some(\n LinkId(\n 0,\n ),\n ),\n speed: Speed40G,\n },\n },\n },\n} file = sled-agent/src/bootstrap/early_networking.rs:436
5212024-08-30T19:40:56.523ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
5222024-08-30T19:40:56.576ZINFOSledAgent (StorageManager): Received DetectedRawDisksChanged { raw_disks: {}, tx: ... } file = sled-storage/src/manager.rs:451
5232024-08-30T19:40:56.576ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
5242024-08-30T19:40:56.576ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
5252024-08-30T19:40:56.576ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
5262024-08-30T19:40:56.576ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
5272024-08-30T19:40:56.576ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
5282024-08-30T19:40:56.576ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
5292024-08-30T19:40:56.576ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
5302024-08-30T19:40:56.576ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
5312024-08-30T19:40:56.576ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
5322024-08-30T19:40:56.576ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
5332024-08-30T19:40:56.576ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
5342024-08-30T19:40:56.576ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
5352024-08-30T19:40:56.576ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
5362024-08-30T19:40:56.576ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
5372024-08-30T19:40:56.576ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
5382024-08-30T19:40:56.576ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
5392024-08-30T19:40:56.576ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
5402024-08-30T19:40:56.576ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
5412024-08-30T19:40:56.576ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
5422024-08-30T19:40:56.576ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
5432024-08-30T19:40:56.576ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
5442024-08-30T19:40:56.576ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
5452024-08-30T19:40:56.734ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_b5d7b957-0082-4dc3-b742-d6e7150a43c2") failed: Property not found. retry in 1.441461019s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_b5d7b957-0082-4dc3-b742-d6e7150a43c2
5462024-08-30T19:40:57.388ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_8b642bba-10a6-428b-9af7-48d5f8622e03") failed: Property not found. retry in 1.225727685s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_8b642bba-10a6-428b-9af7-48d5f8622e03
5472024-08-30T19:40:57.699ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c4910af2-5760-4c5f-bb62-3b2357eab858") failed: Property not found. retry in 1.340277734s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c4910af2-5760-4c5f-bb62-3b2357eab858
5482024-08-30T19:40:59.058ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_internal_dns_b5d7b957-0082-4dc3-b742-d6e7150a43c2
5492024-08-30T19:40:59.058ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_internal_dns_8b642bba-10a6-428b-9af7-48d5f8622e03
5502024-08-30T19:40:59.058ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_internal_dns_c4910af2-5760-4c5f-bb62-3b2357eab858
5512024-08-30T19:40:59.058ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/fc9e5708-cd50-4950-8336-db011df04e25/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
5522024-08-30T19:40:59.058ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/17afe9d4-47cb-4799-8d3d-ad69badedc16/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
5532024-08-30T19:40:59.059ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8635476 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:60102 req_id = 696ad00e-e4a7-42f1-ad05-e3249b65fbc8 response_code = 204 uri = /omicron-zones
5542024-08-30T19:40:59.060ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:1::1]:5353 file = sled-agent/src/rack_setup/service.rs:555
5552024-08-30T19:40:59.120ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:2::1]:5353 file = sled-agent/src/rack_setup/service.rs:555
5562024-08-30T19:40:59.165ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:3::1]:5353 file = sled-agent/src/rack_setup/service.rs:555
5572024-08-30T19:40:59.213ZINFOSledAgent (RSS): Configured all DNS servers file = sled-agent/src/rack_setup/service.rs:591
5582024-08-30T19:40:59.213ZINFOSledAgent (RSS): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
5592024-08-30T19:40:59.245ZINFOSledAgent (RSS): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
5602024-08-30T19:40:59.246ZINFOSledAgent (RSS): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
5612024-08-30T19:40:59.246ZINFOSledAgent (RSS): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
5622024-08-30T19:40:59.276ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(3), zones: [OmicronZoneConfig { id: 8b642bba-10a6-428b-9af7-48d5f8622e03, underlay_address: fd00:1122:3344:1::1, filesystem_pool: Some(ZpoolName { id: 83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: c4910af2-5760-4c5f-bb62-3b2357eab858, underlay_address: fd00:1122:3344:2::1, filesystem_pool: Some(ZpoolName { id: 1693cbb3-d027-4679-ba05-f26fe21ef04a (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 1693cbb3-d027-4679-ba05-f26fe21ef04a (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: b5d7b957-0082-4dc3-b742-d6e7150a43c2, underlay_address: fd00:1122:3344:3::1, filesystem_pool: Some(ZpoolName { id: 161cdb67-d423-452c-be58-7bca2891066e (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 161cdb67-d423-452c-be58-7bca2891066e (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: f4b05376-b0e0-43c5-a4ab-47e66a2a3b7f, underlay_address: fd00:1122:3344:101::1b, filesystem_pool: Some(ZpoolName { id: a2ff29f8-f12c-4d2f-add0-1d5faf6635ee (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 31a2b964-84b7-42d0-9efa-07166b3effd6, kind: Service { id: f4b05376-b0e0-43c5-a4ab-47e66a2a3b7f }, name: Name("ntp-f4b05376-b0e0-43c5-a4ab-47e66a2a3b7f"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 245, 29])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } } }] } file = sled-agent/src/rack_setup/service.rs:438 sled_address = [fd00:1122:3344:101::1]:12345
5632024-08-30T19:40:59.276ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:32834
5642024-08-30T19:40:59.276ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 8b642bba-10a6-428b-9af7-48d5f8622e03, dataset_name: DatasetName { pool_name: ZpoolName { id: 83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
5652024-08-30T19:40:59.276ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 8b642bba-10a6-428b-9af7-48d5f8622e03, dataset_name: DatasetName { pool_name: ZpoolName { id: 83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 (zpool), kind: External }, kind: InternalDns }, responder: ... }
5662024-08-30T19:40:59.298ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c4910af2-5760-4c5f-bb62-3b2357eab858, dataset_name: DatasetName { pool_name: ZpoolName { id: 1693cbb3-d027-4679-ba05-f26fe21ef04a (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
5672024-08-30T19:40:59.298ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: c4910af2-5760-4c5f-bb62-3b2357eab858, dataset_name: DatasetName { pool_name: ZpoolName { id: 1693cbb3-d027-4679-ba05-f26fe21ef04a (zpool), kind: External }, kind: InternalDns }, responder: ... }
5682024-08-30T19:40:59.315ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: b5d7b957-0082-4dc3-b742-d6e7150a43c2, dataset_name: DatasetName { pool_name: ZpoolName { id: 161cdb67-d423-452c-be58-7bca2891066e (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
5692024-08-30T19:40:59.315ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: b5d7b957-0082-4dc3-b742-d6e7150a43c2, dataset_name: DatasetName { pool_name: ZpoolName { id: 161cdb67-d423-452c-be58-7bca2891066e (zpool), kind: External }, kind: InternalDns }, responder: ... }
5702024-08-30T19:40:59.332ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
5712024-08-30T19:40:59.332ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/fc9e5708-cd50-4950-8336-db011df04e25/config/omicron-zones.json file = common/src/ledger.rs:177
5722024-08-30T19:40:59.332ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/17afe9d4-47cb-4799-8d3d-ad69badedc16/config/omicron-zones.json file = common/src/ledger.rs:177
5732024-08-30T19:40:59.333ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
5742024-08-30T19:40:59.343ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
5752024-08-30T19:40:59.376ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
5762024-08-30T19:40:59.377ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
5772024-08-30T19:40:59.377ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
5782024-08-30T19:40:59.407ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte0", ip: 172.30.3.5, mac: MacAddr6([168, 64, 37, 255, 245, 29]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.3.1 } }) }
5792024-08-30T19:40:59.407ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1234 zone_type = boundary_ntp
5802024-08-30T19:40:59.407ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.1.169 file = clients/dpd-client/src/lib.rs:83
5812024-08-30T19:40:59.408ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.1.169 file = clients/dpd-client/src/lib.rs:90
5822024-08-30T19:40:59.408ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.1.169 file = clients/dpd-client/src/lib.rs:118
5832024-08-30T19:40:59.408ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
5842024-08-30T19:40:59.432ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_ntp_f4b05376-b0e0-43c5-a4ab-47e66a2a3b7f file = illumos-utils/src/zone.rs:316
5852024-08-30T19:40:59.449ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_ntp_f4b05376-b0e0-43c5-a4ab-47e66a2a3b7f file = illumos-utils/src/zone.rs:354
5862024-08-30T19:41:00.096ZINFOSledAgent (ServiceManager): Profile for oxz_ntp_f4b05376-b0e0-43c5-a4ab-47e66a2a3b7f: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService4'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::1b'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/chrony-setup"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="allow" value='fd00:1122:3344:100::/56'/> <propval type="boolean" name="boundary" value='true'/> <propval type="astring" name="boundary_pool" value='boundary-ntp.control-plane.oxide.internal'/> <propval type="astring" name="server" value='0.pool.ntp.org'/> </property_group> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="1.1.1.1" /> <value_node value="9.9.9.9" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/ntp"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte0'/> <propval type="astring" name="gateway" value='172.30.3.1'/> <propval type="astring" name="ip" value='172.30.3.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
5872024-08-30T19:41:00.097ZINFOSledAgent (ServiceManager): Booting oxz_ntp_f4b05376-b0e0-43c5-a4ab-47e66a2a3b7f zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_ntp_f4b05376-b0e0-43c5-a4ab-47e66a2a3b7f
5882024-08-30T19:41:00.229ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_f4b05376-b0e0-43c5-a4ab-47e66a2a3b7f") failed: Property not found. retry in 67.940863ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_f4b05376-b0e0-43c5-a4ab-47e66a2a3b7f
5892024-08-30T19:41:00.312ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_f4b05376-b0e0-43c5-a4ab-47e66a2a3b7f") failed: Property not found. retry in 63.770374ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_f4b05376-b0e0-43c5-a4ab-47e66a2a3b7f
5902024-08-30T19:41:00.395ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_f4b05376-b0e0-43c5-a4ab-47e66a2a3b7f") failed: Property not found. retry in 232.712958ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_f4b05376-b0e0-43c5-a4ab-47e66a2a3b7f
5912024-08-30T19:41:00.638ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_f4b05376-b0e0-43c5-a4ab-47e66a2a3b7f") failed: Property not found. retry in 364.936058ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_f4b05376-b0e0-43c5-a4ab-47e66a2a3b7f
5922024-08-30T19:41:01.011ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_f4b05376-b0e0-43c5-a4ab-47e66a2a3b7f") failed: Property not found. retry in 718.619014ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_f4b05376-b0e0-43c5-a4ab-47e66a2a3b7f
5932024-08-30T19:41:01.740ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_f4b05376-b0e0-43c5-a4ab-47e66a2a3b7f") failed: Property not found. retry in 1.193620474s file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_f4b05376-b0e0-43c5-a4ab-47e66a2a3b7f
5942024-08-30T19:41:02.944ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_f4b05376-b0e0-43c5-a4ab-47e66a2a3b7f") failed: Property not found. retry in 999.39296ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_f4b05376-b0e0-43c5-a4ab-47e66a2a3b7f
5952024-08-30T19:41:03.456ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/289c9d10-fa69-4638-aac7-3cfc6b4e23be): error trying to connect: dns error: no record found for Query { name: Name("_nexus._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } file = sled-agent/src/nexus.rs:571 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be
5962024-08-30T19:41:03.953ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_f4b05376-b0e0-43c5-a4ab-47e66a2a3b7f") failed: Property not found. retry in 652.319437ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_f4b05376-b0e0-43c5-a4ab-47e66a2a3b7f
5972024-08-30T19:41:04.615ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_f4b05376-b0e0-43c5-a4ab-47e66a2a3b7f") failed: Property not found. retry in 1.308039314s file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_f4b05376-b0e0-43c5-a4ab-47e66a2a3b7f
5982024-08-30T19:41:05.943ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_ntp_f4b05376-b0e0-43c5-a4ab-47e66a2a3b7f
5992024-08-30T19:41:05.943ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/fc9e5708-cd50-4950-8336-db011df04e25/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
6002024-08-30T19:41:05.944ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/17afe9d4-47cb-4799-8d3d-ad69badedc16/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
6012024-08-30T19:41:05.944ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 6668155 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:32834 req_id = a39ff1cf-b468-4a52-878c-03f6e962c765 response_code = 204 uri = /omicron-zones
6022024-08-30T19:41:05.946ZINFOSledAgent (RSS): Waiting for rack time synchronization file = sled-agent/src/rack_setup/service.rs:632
6032024-08-30T19:41:05.977ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6042024-08-30T19:41:05.977ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:64163
6052024-08-30T19:41:05.985ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_f4b05376-b0e0-43c5-a4ab-47e66a2a3b7f': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3852
6062024-08-30T19:41:05.985ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:902 latency_us = 7999 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:64163 req_id = 0369d7a9-9b9e-4662-83b7-172d580715ff response_code = 500 uri = /timesync
6072024-08-30T19:41:05.987ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6082024-08-30T19:41:06.121ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6092024-08-30T19:41:06.121ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:43446
6102024-08-30T19:41:10.130ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_f4b05376-b0e0-43c5-a4ab-47e66a2a3b7f': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3852
6112024-08-30T19:41:10.130ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:902 latency_us = 4008745 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:43446 req_id = cf15acc0-b2c2-43e4-a985-ccd9e1f5481c response_code = 500 uri = /timesync
6122024-08-30T19:41:10.133ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6132024-08-30T19:41:10.323ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6142024-08-30T19:41:10.323ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:46366
6152024-08-30T19:41:10.332ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8825 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:46366 req_id = 3af8fee5-c7c5-4497-9332-e4effc1de54b response_code = 200 uri = /timesync
6162024-08-30T19:41:10.335ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:640
6172024-08-30T19:41:10.335ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6182024-08-30T19:41:10.579ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6192024-08-30T19:41:10.579ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:38929
6202024-08-30T19:41:10.587ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8096 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:38929 req_id = 0400b2be-e9fd-4162-9cdc-6f649c571e8b response_code = 200 uri = /timesync
6212024-08-30T19:41:10.590ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:640
6222024-08-30T19:41:10.590ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6232024-08-30T19:41:10.869ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6242024-08-30T19:41:10.869ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:33116
6252024-08-30T19:41:10.878ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8471 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:33116 req_id = 64b19e6f-2b95-49be-a6b9-a8c7b88a81fc response_code = 200 uri = /timesync
6262024-08-30T19:41:10.880ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:640
6272024-08-30T19:41:10.880ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6282024-08-30T19:41:11.126ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6292024-08-30T19:41:11.126ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:56185
6302024-08-30T19:41:11.134ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8277 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:56185 req_id = ebccc535-f41e-4c2b-8e64-6439419742c8 response_code = 200 uri = /timesync
6312024-08-30T19:41:11.135ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:640
6322024-08-30T19:41:11.136ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6332024-08-30T19:41:11.489ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6342024-08-30T19:41:11.489ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:56820
6352024-08-30T19:41:11.497ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 7932 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:56820 req_id = eb86dc85-59ef-4fd2-9423-646f5447513b response_code = 200 uri = /timesync
6362024-08-30T19:41:11.500ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:640
6372024-08-30T19:41:11.500ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6382024-08-30T19:41:12.242ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6392024-08-30T19:41:12.242ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:63974
6402024-08-30T19:41:12.250ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 7717 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:63974 req_id = 50e2f0d9-b151-40fd-b104-802d9008b4d3 response_code = 200 uri = /timesync
6412024-08-30T19:41:12.252ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: -0.083477087 } file = sled-agent/src/rack_setup/service.rs:640
6422024-08-30T19:41:12.252ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6432024-08-30T19:41:12.765ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6442024-08-30T19:41:12.765ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:56569
6452024-08-30T19:41:12.773ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8566 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:56569 req_id = 23359025-10d0-42eb-93f5-7a861aa2ba19 response_code = 200 uri = /timesync
6462024-08-30T19:41:12.777ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: -0.082054965 } file = sled-agent/src/rack_setup/service.rs:640
6472024-08-30T19:41:12.777ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6482024-08-30T19:41:13.374ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6492024-08-30T19:41:13.375ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:42055
6502024-08-30T19:41:13.382ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 7325 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:42055 req_id = 64e8cfcf-6fd9-4b8d-ae81-a5c29a3e2a6e response_code = 200 uri = /timesync
6512024-08-30T19:41:13.384ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: -0.088374175 } file = sled-agent/src/rack_setup/service.rs:640
6522024-08-30T19:41:13.384ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6532024-08-30T19:41:13.981ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6542024-08-30T19:41:13.981ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:33093
6552024-08-30T19:41:13.989ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8496 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:33093 req_id = 3d2fa6da-f4a9-4450-962b-182cdbc42a27 response_code = 200 uri = /timesync
6562024-08-30T19:41:13.992ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: -0.086724438 } file = sled-agent/src/rack_setup/service.rs:640
6572024-08-30T19:41:13.992ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6582024-08-30T19:41:14.812ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6592024-08-30T19:41:14.812ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:63512
6602024-08-30T19:41:14.821ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8249 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:63512 req_id = 24c35405-7b1c-49ab-b349-fc9a6d3b2512 response_code = 200 uri = /timesync
6612024-08-30T19:41:14.823ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: -0.083928242 } file = sled-agent/src/rack_setup/service.rs:640
6622024-08-30T19:41:14.823ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6632024-08-30T19:41:15.524ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6642024-08-30T19:41:15.524ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:45021
6652024-08-30T19:41:15.533ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8671 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:45021 req_id = d7fe6383-014d-44ff-81ad-b28ca574c262 response_code = 200 uri = /timesync
6662024-08-30T19:41:15.535ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: -0.082388274 } file = sled-agent/src/rack_setup/service.rs:640
6672024-08-30T19:41:15.535ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6682024-08-30T19:41:16.187ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6692024-08-30T19:41:16.188ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:45278
6702024-08-30T19:41:16.196ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8813 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:45278 req_id = 99b59293-c4d9-4c20-8202-2506b9d9edb9 response_code = 200 uri = /timesync
6712024-08-30T19:41:16.199ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: -0.078662492 } file = sled-agent/src/rack_setup/service.rs:640
6722024-08-30T19:41:16.199ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6732024-08-30T19:41:17.219ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6742024-08-30T19:41:17.220ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:40812
6752024-08-30T19:41:17.228ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8280 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:40812 req_id = f24b915c-fb78-47a6-98f3-391f471e4e1c response_code = 200 uri = /timesync
6762024-08-30T19:41:17.230ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1725046875.3762553, correction: -0.076171748 } file = sled-agent/src/rack_setup/service.rs:640
6772024-08-30T19:41:17.230ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6782024-08-30T19:41:17.646ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 356.028247ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be
6792024-08-30T19:41:18.376ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6802024-08-30T19:41:18.376ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:42871
6812024-08-30T19:41:18.385ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8354 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:42871 req_id = 1f54e9ec-906f-4546-a66d-b933c65e5be1 response_code = 200 uri = /timesync
6822024-08-30T19:41:18.387ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 3638887493, ip_addr: 216.229.4.69, stratum: 3, ref_time: 1725046877.5688891, correction: -0.073041998 } file = sled-agent/src/rack_setup/service.rs:640
6832024-08-30T19:41:18.387ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6842024-08-30T19:41:20.457ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6852024-08-30T19:41:20.457ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:47632
6862024-08-30T19:41:20.466ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8424 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:47632 req_id = 993e13eb-222b-4d7c-a6d9-e659dbb45b79 response_code = 200 uri = /timesync
6872024-08-30T19:41:20.468ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 3638887493, ip_addr: 216.229.4.69, stratum: 3, ref_time: 1725046879.1931322, correction: -0.066785522 } file = sled-agent/src/rack_setup/service.rs:640
6882024-08-30T19:41:20.468ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6892024-08-30T19:41:23.068ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6902024-08-30T19:41:23.068ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:36357
6912024-08-30T19:41:23.079ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 10804 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:36357 req_id = e0feb348-93c2-4553-9c6c-1397ccc8ebce response_code = 200 uri = /timesync
6922024-08-30T19:41:23.082ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 3638887493, ip_addr: 216.229.4.69, stratum: 3, ref_time: 1725046881.215153, correction: -0.059413396 } file = sled-agent/src/rack_setup/service.rs:640
6932024-08-30T19:41:23.082ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6942024-08-30T19:41:25.976ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6952024-08-30T19:41:25.976ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:54480
6962024-08-30T19:41:25.985ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8552 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:54480 req_id = 38a313f0-4a55-47f6-80e2-c5d485de6f90 response_code = 200 uri = /timesync
6972024-08-30T19:41:25.987ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 3638887493, ip_addr: 216.229.4.69, stratum: 3, ref_time: 1725046884.4027438, correction: -0.051530544 } file = sled-agent/src/rack_setup/service.rs:640
6982024-08-30T19:41:25.987ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6992024-08-30T19:41:26.575ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:36359
7002024-08-30T19:41:26.576ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 56 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:36359 req_id = 1c1b7f01-eb23-4f6e-8279-e78309c4c586 response_code = 200 uri = /baseboard
7012024-08-30T19:41:27.793ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
7022024-08-30T19:41:27.793ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:53946
7032024-08-30T19:41:27.814ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 20935 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:53946 req_id = f366049e-58eb-4a3d-b43d-00d0dcd4e9e1 response_code = 200 uri = /timesync
7042024-08-30T19:41:27.817ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: true, ref_id: 3638887493, ip_addr: 216.229.4.69, stratum: 3, ref_time: 1725046886.5172324, correction: -0.046569932 } file = sled-agent/src/rack_setup/service.rs:640
7052024-08-30T19:41:27.817ZINFOSledAgent (RSS): Finished setting up Internal DNS and NTP file = sled-agent/src/rack_setup/service.rs:1266
7062024-08-30T19:41:27.847ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 8b642bba-10a6-428b-9af7-48d5f8622e03, underlay_address: fd00:1122:3344:1::1, filesystem_pool: Some(ZpoolName { id: 83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: c4910af2-5760-4c5f-bb62-3b2357eab858, underlay_address: fd00:1122:3344:2::1, filesystem_pool: Some(ZpoolName { id: 1693cbb3-d027-4679-ba05-f26fe21ef04a (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 1693cbb3-d027-4679-ba05-f26fe21ef04a (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: b5d7b957-0082-4dc3-b742-d6e7150a43c2, underlay_address: fd00:1122:3344:3::1, filesystem_pool: Some(ZpoolName { id: 161cdb67-d423-452c-be58-7bca2891066e (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 161cdb67-d423-452c-be58-7bca2891066e (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: f4b05376-b0e0-43c5-a4ab-47e66a2a3b7f, underlay_address: fd00:1122:3344:101::1b, filesystem_pool: Some(ZpoolName { id: a2ff29f8-f12c-4d2f-add0-1d5faf6635ee (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 31a2b964-84b7-42d0-9efa-07166b3effd6, kind: Service { id: f4b05376-b0e0-43c5-a4ab-47e66a2a3b7f }, name: Name("ntp-f4b05376-b0e0-43c5-a4ab-47e66a2a3b7f"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 245, 29])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: e5ae20e7-61c5-4df1-8e9c-36c9a79e1c3e, underlay_address: fd00:1122:3344:101::3, filesystem_pool: Some(ZpoolName { id: 83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 (zpool), kind: External } } } }, OmicronZoneConfig { id: a00a8432-4fac-47a4-9bee-ade776eac084, underlay_address: fd00:1122:3344:101::4, filesystem_pool: Some(ZpoolName { id: 1693cbb3-d027-4679-ba05-f26fe21ef04a (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 1693cbb3-d027-4679-ba05-f26fe21ef04a (zpool), kind: External } } } }, OmicronZoneConfig { id: 0770f0cb-550c-44e1-bda1-c4a6a97fdce7, underlay_address: fd00:1122:3344:101::5, filesystem_pool: Some(ZpoolName { id: 161cdb67-d423-452c-be58-7bca2891066e (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 161cdb67-d423-452c-be58-7bca2891066e (zpool), kind: External } } } }, OmicronZoneConfig { id: 47fef55b-2e70-4ed3-83d5-ed6aeeae8bc8, underlay_address: fd00:1122:3344:101::6, filesystem_pool: Some(ZpoolName { id: 3fd4f884-7e56-4c58-871f-b010d32df20b (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 3fd4f884-7e56-4c58-871f-b010d32df20b (zpool), kind: External } } } }, OmicronZoneConfig { id: 26f53738-d5c9-44ca-8ef9-8a6621900268, underlay_address: fd00:1122:3344:101::7, filesystem_pool: Some(ZpoolName { id: 6113cfc6-d07f-470a-963b-4465a62b65f1 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 6113cfc6-d07f-470a-963b-4465a62b65f1 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:438 sled_address = [fd00:1122:3344:101::1]:12345
7072024-08-30T19:41:27.847ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:57340
7082024-08-30T19:41:27.848ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 8b642bba-10a6-428b-9af7-48d5f8622e03, dataset_name: DatasetName { pool_name: ZpoolName { id: 83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
7092024-08-30T19:41:27.848ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 8b642bba-10a6-428b-9af7-48d5f8622e03, dataset_name: DatasetName { pool_name: ZpoolName { id: 83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 (zpool), kind: External }, kind: InternalDns }, responder: ... }
7102024-08-30T19:41:27.869ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c4910af2-5760-4c5f-bb62-3b2357eab858, dataset_name: DatasetName { pool_name: ZpoolName { id: 1693cbb3-d027-4679-ba05-f26fe21ef04a (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
7112024-08-30T19:41:27.869ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: c4910af2-5760-4c5f-bb62-3b2357eab858, dataset_name: DatasetName { pool_name: ZpoolName { id: 1693cbb3-d027-4679-ba05-f26fe21ef04a (zpool), kind: External }, kind: InternalDns }, responder: ... }
7122024-08-30T19:41:27.888ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: b5d7b957-0082-4dc3-b742-d6e7150a43c2, dataset_name: DatasetName { pool_name: ZpoolName { id: 161cdb67-d423-452c-be58-7bca2891066e (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
7132024-08-30T19:41:27.888ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: b5d7b957-0082-4dc3-b742-d6e7150a43c2, dataset_name: DatasetName { pool_name: ZpoolName { id: 161cdb67-d423-452c-be58-7bca2891066e (zpool), kind: External }, kind: InternalDns }, responder: ... }
7142024-08-30T19:41:27.906ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: e5ae20e7-61c5-4df1-8e9c-36c9a79e1c3e, dataset_name: DatasetName { pool_name: ZpoolName { id: 83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
7152024-08-30T19:41:27.906ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: e5ae20e7-61c5-4df1-8e9c-36c9a79e1c3e, dataset_name: DatasetName { pool_name: ZpoolName { id: 83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 (zpool), kind: External }, kind: Cockroach }, responder: ... }
7162024-08-30T19:41:27.958ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: a00a8432-4fac-47a4-9bee-ade776eac084, dataset_name: DatasetName { pool_name: ZpoolName { id: 1693cbb3-d027-4679-ba05-f26fe21ef04a (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
7172024-08-30T19:41:27.958ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: a00a8432-4fac-47a4-9bee-ade776eac084, dataset_name: DatasetName { pool_name: ZpoolName { id: 1693cbb3-d027-4679-ba05-f26fe21ef04a (zpool), kind: External }, kind: Cockroach }, responder: ... }
7182024-08-30T19:41:28.008ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 0770f0cb-550c-44e1-bda1-c4a6a97fdce7, dataset_name: DatasetName { pool_name: ZpoolName { id: 161cdb67-d423-452c-be58-7bca2891066e (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
7192024-08-30T19:41:28.008ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 0770f0cb-550c-44e1-bda1-c4a6a97fdce7, dataset_name: DatasetName { pool_name: ZpoolName { id: 161cdb67-d423-452c-be58-7bca2891066e (zpool), kind: External }, kind: Cockroach }, responder: ... }
7202024-08-30T19:41:28.059ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 47fef55b-2e70-4ed3-83d5-ed6aeeae8bc8, dataset_name: DatasetName { pool_name: ZpoolName { id: 3fd4f884-7e56-4c58-871f-b010d32df20b (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
7212024-08-30T19:41:28.059ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 47fef55b-2e70-4ed3-83d5-ed6aeeae8bc8, dataset_name: DatasetName { pool_name: ZpoolName { id: 3fd4f884-7e56-4c58-871f-b010d32df20b (zpool), kind: External }, kind: Cockroach }, responder: ... }
7222024-08-30T19:41:28.109ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 26f53738-d5c9-44ca-8ef9-8a6621900268, dataset_name: DatasetName { pool_name: ZpoolName { id: 6113cfc6-d07f-470a-963b-4465a62b65f1 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
7232024-08-30T19:41:28.109ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 26f53738-d5c9-44ca-8ef9-8a6621900268, dataset_name: DatasetName { pool_name: ZpoolName { id: 6113cfc6-d07f-470a-963b-4465a62b65f1 (zpool), kind: External }, kind: Cockroach }, responder: ... }
7242024-08-30T19:41:28.163ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
7252024-08-30T19:41:28.163ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/fc9e5708-cd50-4950-8336-db011df04e25/config/omicron-zones.json file = common/src/ledger.rs:177
7262024-08-30T19:41:28.163ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/17afe9d4-47cb-4799-8d3d-ad69badedc16/config/omicron-zones.json file = common/src/ledger.rs:177
7272024-08-30T19:41:28.163ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
7282024-08-30T19:41:28.243ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
7292024-08-30T19:41:28.243ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
7302024-08-30T19:41:28.243ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
7312024-08-30T19:41:28.243ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
7322024-08-30T19:41:28.243ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
7332024-08-30T19:41:28.337ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_47fef55b-2e70-4ed3-83d5-ed6aeeae8bc8 file = illumos-utils/src/zone.rs:316
7342024-08-30T19:41:28.341ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_26f53738-d5c9-44ca-8ef9-8a6621900268 file = illumos-utils/src/zone.rs:316
7352024-08-30T19:41:28.345ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_e5ae20e7-61c5-4df1-8e9c-36c9a79e1c3e file = illumos-utils/src/zone.rs:316
7362024-08-30T19:41:28.350ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_0770f0cb-550c-44e1-bda1-c4a6a97fdce7 file = illumos-utils/src/zone.rs:316
7372024-08-30T19:41:28.354ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_a00a8432-4fac-47a4-9bee-ade776eac084 file = illumos-utils/src/zone.rs:316
7382024-08-30T19:41:28.359ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_47fef55b-2e70-4ed3-83d5-ed6aeeae8bc8 file = illumos-utils/src/zone.rs:354
7392024-08-30T19:41:28.364ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_26f53738-d5c9-44ca-8ef9-8a6621900268 file = illumos-utils/src/zone.rs:354
7402024-08-30T19:41:28.368ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_e5ae20e7-61c5-4df1-8e9c-36c9a79e1c3e file = illumos-utils/src/zone.rs:354
7412024-08-30T19:41:28.373ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_0770f0cb-550c-44e1-bda1-c4a6a97fdce7 file = illumos-utils/src/zone.rs:354
7422024-08-30T19:41:28.377ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_a00a8432-4fac-47a4-9bee-ade776eac084 file = illumos-utils/src/zone.rs:354
7432024-08-30T19:41:31.306ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_47fef55b-2e70-4ed3-83d5-ed6aeeae8bc8: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService5'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::6]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='47fef55b-2e70-4ed3-83d5-ed6aeeae8bc8'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::6]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::6]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
7442024-08-30T19:41:31.306ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_47fef55b-2e70-4ed3-83d5-ed6aeeae8bc8 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_47fef55b-2e70-4ed3-83d5-ed6aeeae8bc8
7452024-08-30T19:41:31.312ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_26f53738-d5c9-44ca-8ef9-8a6621900268: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService6'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::7'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::7]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='26f53738-d5c9-44ca-8ef9-8a6621900268'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::7]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::7]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
7462024-08-30T19:41:31.312ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_26f53738-d5c9-44ca-8ef9-8a6621900268 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_26f53738-d5c9-44ca-8ef9-8a6621900268
7472024-08-30T19:41:31.374ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_a00a8432-4fac-47a4-9bee-ade776eac084: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService9'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::4'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::4]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='a00a8432-4fac-47a4-9bee-ade776eac084'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::4]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::4]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
7482024-08-30T19:41:31.374ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_a00a8432-4fac-47a4-9bee-ade776eac084 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_a00a8432-4fac-47a4-9bee-ade776eac084
7492024-08-30T19:41:31.437ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_0770f0cb-550c-44e1-bda1-c4a6a97fdce7: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService8'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::5]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='0770f0cb-550c-44e1-bda1-c4a6a97fdce7'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::5]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::5]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
7502024-08-30T19:41:31.437ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_0770f0cb-550c-44e1-bda1-c4a6a97fdce7 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_0770f0cb-550c-44e1-bda1-c4a6a97fdce7
7512024-08-30T19:41:31.454ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_47fef55b-2e70-4ed3-83d5-ed6aeeae8bc8") failed: Property not found. retry in 34.196889ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_47fef55b-2e70-4ed3-83d5-ed6aeeae8bc8
7522024-08-30T19:41:31.454ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_e5ae20e7-61c5-4df1-8e9c-36c9a79e1c3e: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService7'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::3'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::3]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='e5ae20e7-61c5-4df1-8e9c-36c9a79e1c3e'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::3]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::3]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
7532024-08-30T19:41:31.462ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_26f53738-d5c9-44ca-8ef9-8a6621900268") failed: Property not found. retry in 40.363922ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_26f53738-d5c9-44ca-8ef9-8a6621900268
7542024-08-30T19:41:31.462ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_e5ae20e7-61c5-4df1-8e9c-36c9a79e1c3e zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_e5ae20e7-61c5-4df1-8e9c-36c9a79e1c3e
7552024-08-30T19:41:31.497ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_47fef55b-2e70-4ed3-83d5-ed6aeeae8bc8") failed: Property not found. retry in 60.921555ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_47fef55b-2e70-4ed3-83d5-ed6aeeae8bc8
7562024-08-30T19:41:31.509ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a00a8432-4fac-47a4-9bee-ade776eac084") failed: Property not found. retry in 74.792079ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a00a8432-4fac-47a4-9bee-ade776eac084
7572024-08-30T19:41:31.522ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_26f53738-d5c9-44ca-8ef9-8a6621900268") failed: Property not found. retry in 111.150747ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_26f53738-d5c9-44ca-8ef9-8a6621900268
7582024-08-30T19:41:31.616ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_47fef55b-2e70-4ed3-83d5-ed6aeeae8bc8") failed: Property not found. retry in 242.488633ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_47fef55b-2e70-4ed3-83d5-ed6aeeae8bc8
7592024-08-30T19:41:31.631ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a00a8432-4fac-47a4-9bee-ade776eac084") failed: Property not found. retry in 126.27695ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a00a8432-4fac-47a4-9bee-ade776eac084
7602024-08-30T19:41:31.644ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_26f53738-d5c9-44ca-8ef9-8a6621900268") failed: Property not found. retry in 259.098346ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_26f53738-d5c9-44ca-8ef9-8a6621900268
7612024-08-30T19:41:31.837ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a00a8432-4fac-47a4-9bee-ade776eac084") failed: Property not found. retry in 288.13587ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a00a8432-4fac-47a4-9bee-ade776eac084
7622024-08-30T19:41:31.925ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e5ae20e7-61c5-4df1-8e9c-36c9a79e1c3e") failed: Property not found. retry in 55.72726ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e5ae20e7-61c5-4df1-8e9c-36c9a79e1c3e
7632024-08-30T19:41:32.198ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0770f0cb-550c-44e1-bda1-c4a6a97fdce7") failed: Property not found. retry in 50.498749ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0770f0cb-550c-44e1-bda1-c4a6a97fdce7
7642024-08-30T19:41:32.213ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_47fef55b-2e70-4ed3-83d5-ed6aeeae8bc8") failed: Property not found. retry in 289.546408ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_47fef55b-2e70-4ed3-83d5-ed6aeeae8bc8
7652024-08-30T19:41:32.556ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_26f53738-d5c9-44ca-8ef9-8a6621900268") failed: Property not found. retry in 518.073256ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_26f53738-d5c9-44ca-8ef9-8a6621900268
7662024-08-30T19:41:32.615ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e5ae20e7-61c5-4df1-8e9c-36c9a79e1c3e") failed: Property not found. retry in 142.983167ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e5ae20e7-61c5-4df1-8e9c-36c9a79e1c3e
7672024-08-30T19:41:32.956ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0770f0cb-550c-44e1-bda1-c4a6a97fdce7") failed: Property not found. retry in 116.449225ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0770f0cb-550c-44e1-bda1-c4a6a97fdce7
7682024-08-30T19:41:33.318ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 369.219038ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be
7692024-08-30T19:41:33.402ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a00a8432-4fac-47a4-9bee-ade776eac084") failed: Property not found. retry in 291.34363ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a00a8432-4fac-47a4-9bee-ade776eac084
7702024-08-30T19:41:33.720ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_47fef55b-2e70-4ed3-83d5-ed6aeeae8bc8") failed: Property not found. retry in 598.655068ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_47fef55b-2e70-4ed3-83d5-ed6aeeae8bc8
7712024-08-30T19:41:33.963ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_26f53738-d5c9-44ca-8ef9-8a6621900268") failed: Property not found. retry in 571.966609ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_26f53738-d5c9-44ca-8ef9-8a6621900268
7722024-08-30T19:41:34.022ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e5ae20e7-61c5-4df1-8e9c-36c9a79e1c3e") failed: Property not found. retry in 200.470744ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e5ae20e7-61c5-4df1-8e9c-36c9a79e1c3e
7732024-08-30T19:41:34.037ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0770f0cb-550c-44e1-bda1-c4a6a97fdce7") failed: Property not found. retry in 291.486141ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0770f0cb-550c-44e1-bda1-c4a6a97fdce7
7742024-08-30T19:41:34.049ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a00a8432-4fac-47a4-9bee-ade776eac084") failed: Property not found. retry in 721.173043ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a00a8432-4fac-47a4-9bee-ade776eac084
7752024-08-30T19:41:34.233ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e5ae20e7-61c5-4df1-8e9c-36c9a79e1c3e") failed: Property not found. retry in 211.034475ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e5ae20e7-61c5-4df1-8e9c-36c9a79e1c3e
7762024-08-30T19:41:34.329ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_47fef55b-2e70-4ed3-83d5-ed6aeeae8bc8") failed: Property not found. retry in 1.41393317s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_47fef55b-2e70-4ed3-83d5-ed6aeeae8bc8
7772024-08-30T19:41:34.338ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0770f0cb-550c-44e1-bda1-c4a6a97fdce7") failed: Property not found. retry in 493.873801ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0770f0cb-550c-44e1-bda1-c4a6a97fdce7
7782024-08-30T19:41:34.456ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e5ae20e7-61c5-4df1-8e9c-36c9a79e1c3e") failed: Property not found. retry in 615.475189ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e5ae20e7-61c5-4df1-8e9c-36c9a79e1c3e
7792024-08-30T19:41:34.542ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_26f53738-d5c9-44ca-8ef9-8a6621900268") failed: Property not found. retry in 544.334468ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_26f53738-d5c9-44ca-8ef9-8a6621900268
7802024-08-30T19:41:34.779ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a00a8432-4fac-47a4-9bee-ade776eac084") failed: Property not found. retry in 970.915124ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a00a8432-4fac-47a4-9bee-ade776eac084
7812024-08-30T19:41:34.839ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0770f0cb-550c-44e1-bda1-c4a6a97fdce7") failed: Property not found. retry in 1.024595673s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0770f0cb-550c-44e1-bda1-c4a6a97fdce7
7822024-08-30T19:41:35.079ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e5ae20e7-61c5-4df1-8e9c-36c9a79e1c3e") failed: Property not found. retry in 826.248977ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e5ae20e7-61c5-4df1-8e9c-36c9a79e1c3e
7832024-08-30T19:41:35.093ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_26f53738-d5c9-44ca-8ef9-8a6621900268") failed: Property not found. retry in 1.049257923s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_26f53738-d5c9-44ca-8ef9-8a6621900268
7842024-08-30T19:41:35.748ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_47fef55b-2e70-4ed3-83d5-ed6aeeae8bc8") failed: Property not found. retry in 1.248887551s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_47fef55b-2e70-4ed3-83d5-ed6aeeae8bc8
7852024-08-30T19:41:35.756ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a00a8432-4fac-47a4-9bee-ade776eac084") failed: Property not found. retry in 844.404177ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a00a8432-4fac-47a4-9bee-ade776eac084
7862024-08-30T19:41:35.869ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0770f0cb-550c-44e1-bda1-c4a6a97fdce7") failed: Property not found. retry in 954.626212ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0770f0cb-550c-44e1-bda1-c4a6a97fdce7
7872024-08-30T19:41:35.911ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e5ae20e7-61c5-4df1-8e9c-36c9a79e1c3e") failed: Property not found. retry in 760.097214ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e5ae20e7-61c5-4df1-8e9c-36c9a79e1c3e
7882024-08-30T19:41:36.151ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_26f53738-d5c9-44ca-8ef9-8a6621900268") failed: Property not found. retry in 983.823122ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_26f53738-d5c9-44ca-8ef9-8a6621900268
7892024-08-30T19:41:36.610ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a00a8432-4fac-47a4-9bee-ade776eac084") failed: Property not found. retry in 634.237403ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a00a8432-4fac-47a4-9bee-ade776eac084
7902024-08-30T19:41:36.680ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e5ae20e7-61c5-4df1-8e9c-36c9a79e1c3e") failed: Property not found. retry in 503.85863ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e5ae20e7-61c5-4df1-8e9c-36c9a79e1c3e
7912024-08-30T19:41:36.832ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0770f0cb-550c-44e1-bda1-c4a6a97fdce7") failed: Property not found. retry in 1.13094992s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0770f0cb-550c-44e1-bda1-c4a6a97fdce7
7922024-08-30T19:41:37.003ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_47fef55b-2e70-4ed3-83d5-ed6aeeae8bc8") failed: Property not found. retry in 955.749104ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_47fef55b-2e70-4ed3-83d5-ed6aeeae8bc8
7932024-08-30T19:41:37.141ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_26f53738-d5c9-44ca-8ef9-8a6621900268") failed: Property not found. retry in 855.285344ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_26f53738-d5c9-44ca-8ef9-8a6621900268
7942024-08-30T19:41:37.190ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e5ae20e7-61c5-4df1-8e9c-36c9a79e1c3e") failed: Property not found. retry in 877.948649ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e5ae20e7-61c5-4df1-8e9c-36c9a79e1c3e
7952024-08-30T19:41:37.251ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a00a8432-4fac-47a4-9bee-ade776eac084") failed: Property not found. retry in 603.333194ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a00a8432-4fac-47a4-9bee-ade776eac084
7962024-08-30T19:41:37.860ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a00a8432-4fac-47a4-9bee-ade776eac084") failed: Property not found. retry in 583.574364ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a00a8432-4fac-47a4-9bee-ade776eac084
7972024-08-30T19:41:37.965ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_47fef55b-2e70-4ed3-83d5-ed6aeeae8bc8") failed: Property not found. retry in 805.475229ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_47fef55b-2e70-4ed3-83d5-ed6aeeae8bc8
7982024-08-30T19:41:37.972ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0770f0cb-550c-44e1-bda1-c4a6a97fdce7") failed: Property not found. retry in 564.621649ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0770f0cb-550c-44e1-bda1-c4a6a97fdce7
7992024-08-30T19:41:38.001ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_26f53738-d5c9-44ca-8ef9-8a6621900268") failed: Property not found. retry in 1.451133226s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_26f53738-d5c9-44ca-8ef9-8a6621900268
8002024-08-30T19:41:38.073ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e5ae20e7-61c5-4df1-8e9c-36c9a79e1c3e") failed: Property not found. retry in 587.319916ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e5ae20e7-61c5-4df1-8e9c-36c9a79e1c3e
8012024-08-30T19:41:38.451ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a00a8432-4fac-47a4-9bee-ade776eac084") failed: Property not found. retry in 623.764557ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a00a8432-4fac-47a4-9bee-ade776eac084
8022024-08-30T19:41:38.545ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0770f0cb-550c-44e1-bda1-c4a6a97fdce7") failed: Property not found. retry in 1.36173727s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0770f0cb-550c-44e1-bda1-c4a6a97fdce7
8032024-08-30T19:41:38.667ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e5ae20e7-61c5-4df1-8e9c-36c9a79e1c3e") failed: Property not found. retry in 509.747672ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e5ae20e7-61c5-4df1-8e9c-36c9a79e1c3e
8042024-08-30T19:41:38.779ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_47fef55b-2e70-4ed3-83d5-ed6aeeae8bc8") failed: Property not found. retry in 1.482865431s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_47fef55b-2e70-4ed3-83d5-ed6aeeae8bc8
8052024-08-30T19:41:39.082ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a00a8432-4fac-47a4-9bee-ade776eac084") failed: Property not found. retry in 1.267138228s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a00a8432-4fac-47a4-9bee-ade776eac084
8062024-08-30T19:41:39.186ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e5ae20e7-61c5-4df1-8e9c-36c9a79e1c3e") failed: Property not found. retry in 1.062092306s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e5ae20e7-61c5-4df1-8e9c-36c9a79e1c3e
8072024-08-30T19:41:39.459ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_26f53738-d5c9-44ca-8ef9-8a6621900268") failed: Property not found. retry in 1.365796985s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_26f53738-d5c9-44ca-8ef9-8a6621900268
8082024-08-30T19:41:39.915ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0770f0cb-550c-44e1-bda1-c4a6a97fdce7") failed: Property not found. retry in 531.082256ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0770f0cb-550c-44e1-bda1-c4a6a97fdce7
8092024-08-30T19:41:40.849ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_cockroachdb_47fef55b-2e70-4ed3-83d5-ed6aeeae8bc8
8102024-08-30T19:41:40.849ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_cockroachdb_26f53738-d5c9-44ca-8ef9-8a6621900268
8112024-08-30T19:41:40.849ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_cockroachdb_e5ae20e7-61c5-4df1-8e9c-36c9a79e1c3e
8122024-08-30T19:41:40.849ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_cockroachdb_0770f0cb-550c-44e1-bda1-c4a6a97fdce7
8132024-08-30T19:41:40.849ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_cockroachdb_a00a8432-4fac-47a4-9bee-ade776eac084
8142024-08-30T19:41:40.850ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/fc9e5708-cd50-4950-8336-db011df04e25/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
8152024-08-30T19:41:40.850ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/17afe9d4-47cb-4799-8d3d-ad69badedc16/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
8162024-08-30T19:41:40.850ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 13038633 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:57340 req_id = 98ae9796-93e9-4729-9539-af853c1e507e response_code = 204 uri = /omicron-zones
8172024-08-30T19:41:40.886ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:39068
8182024-08-30T19:41:40.921ZINFOSledAgent (ServiceManager): Initializing CRDB Cluster - sending request to [fd00:1122:3344:101::5]:32221 file = sled-agent/src/services.rs:3686
8192024-08-30T19:42:05.269ZINFOSledAgent (ServiceManager): Formatting CRDB file = sled-agent/src/services.rs:3704
8202024-08-30T19:42:17.204ZINFOSledAgent (ServiceManager): Formatting CRDB - Completed file = sled-agent/src/services.rs:3727
8212024-08-30T19:42:17.206ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 36329692 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:39068 req_id = 512fa6d1-797f-4333-8294-d2682d49df88 response_code = 204 uri = /cockroachdb
8222024-08-30T19:42:17.213ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:60040
8232024-08-30T19:42:17.236ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 1.47065253s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be
8242024-08-30T19:42:17.272ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(5), zones: [OmicronZoneConfig { id: 8b642bba-10a6-428b-9af7-48d5f8622e03, underlay_address: fd00:1122:3344:1::1, filesystem_pool: Some(ZpoolName { id: 83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: c4910af2-5760-4c5f-bb62-3b2357eab858, underlay_address: fd00:1122:3344:2::1, filesystem_pool: Some(ZpoolName { id: 1693cbb3-d027-4679-ba05-f26fe21ef04a (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 1693cbb3-d027-4679-ba05-f26fe21ef04a (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: b5d7b957-0082-4dc3-b742-d6e7150a43c2, underlay_address: fd00:1122:3344:3::1, filesystem_pool: Some(ZpoolName { id: 161cdb67-d423-452c-be58-7bca2891066e (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 161cdb67-d423-452c-be58-7bca2891066e (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: f4b05376-b0e0-43c5-a4ab-47e66a2a3b7f, underlay_address: fd00:1122:3344:101::1b, filesystem_pool: Some(ZpoolName { id: a2ff29f8-f12c-4d2f-add0-1d5faf6635ee (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 31a2b964-84b7-42d0-9efa-07166b3effd6, kind: Service { id: f4b05376-b0e0-43c5-a4ab-47e66a2a3b7f }, name: Name("ntp-f4b05376-b0e0-43c5-a4ab-47e66a2a3b7f"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 245, 29])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: e5ae20e7-61c5-4df1-8e9c-36c9a79e1c3e, underlay_address: fd00:1122:3344:101::3, filesystem_pool: Some(ZpoolName { id: 83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 (zpool), kind: External } } } }, OmicronZoneConfig { id: a00a8432-4fac-47a4-9bee-ade776eac084, underlay_address: fd00:1122:3344:101::4, filesystem_pool: Some(ZpoolName { id: 1693cbb3-d027-4679-ba05-f26fe21ef04a (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 1693cbb3-d027-4679-ba05-f26fe21ef04a (zpool), kind: External } } } }, OmicronZoneConfig { id: 0770f0cb-550c-44e1-bda1-c4a6a97fdce7, underlay_address: fd00:1122:3344:101::5, filesystem_pool: Some(ZpoolName { id: 161cdb67-d423-452c-be58-7bca2891066e (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 161cdb67-d423-452c-be58-7bca2891066e (zpool), kind: External } } } }, OmicronZoneConfig { id: 47fef55b-2e70-4ed3-83d5-ed6aeeae8bc8, underlay_address: fd00:1122:3344:101::6, filesystem_pool: Some(ZpoolName { id: 3fd4f884-7e56-4c58-871f-b010d32df20b (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 3fd4f884-7e56-4c58-871f-b010d32df20b (zpool), kind: External } } } }, OmicronZoneConfig { id: 26f53738-d5c9-44ca-8ef9-8a6621900268, underlay_address: fd00:1122:3344:101::7, filesystem_pool: Some(ZpoolName { id: 6113cfc6-d07f-470a-963b-4465a62b65f1 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 6113cfc6-d07f-470a-963b-4465a62b65f1 (zpool), kind: External } } } }, OmicronZoneConfig { id: ff52340f-525a-4b88-bfc8-1cb0fe9ceba7, underlay_address: fd00:1122:3344:101::8, filesystem_pool: Some(ZpoolName { id: 83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 (zpool), kind: External }), zone_type: ExternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 (zpool), kind: External } }, http_address: [fd00:1122:3344:101::8]:5353, dns_address: 10.151.1.164:53, nic: NetworkInterface { id: e6ac84e1-d9f1-4c03-8dd9-78d5c3b887de, kind: Service { id: ff52340f-525a-4b88-bfc8-1cb0fe9ceba7 }, name: Name("external-dns-ff52340f-525a-4b88-bfc8-1cb0fe9ceba7"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 231])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } } }, OmicronZoneConfig { id: b5636b10-76ec-4aa4-8bd4-1f95780813df, underlay_address: fd00:1122:3344:101::9, filesystem_pool: Some(ZpoolName { id: 1693cbb3-d027-4679-ba05-f26fe21ef04a (zpool), kind: External }), zone_type: ExternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 1693cbb3-d027-4679-ba05-f26fe21ef04a (zpool), kind: External } }, http_address: [fd00:1122:3344:101::9]:5353, dns_address: 10.151.1.165:53, nic: NetworkInterface { id: 148fa4d0-99ac-4f4c-b957-d26cc8730745, kind: Service { id: b5636b10-76ec-4aa4-8bd4-1f95780813df }, name: Name("external-dns-b5636b10-76ec-4aa4-8bd4-1f95780813df"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 151, 107])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } } }, OmicronZoneConfig { id: ba4aef74-7b33-4ded-a46a-dc2534b0db9d, underlay_address: fd00:1122:3344:101::a, filesystem_pool: Some(ZpoolName { id: 3fd4f884-7e56-4c58-871f-b010d32df20b (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::a]:12221, external_ip: 10.151.1.166, nic: NetworkInterface { id: 82aefc08-c1a4-48ce-8225-d8f4409ab816, kind: Service { id: ba4aef74-7b33-4ded-a46a-dc2534b0db9d }, name: Name("nexus-ba4aef74-7b33-4ded-a46a-dc2534b0db9d"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 158, 211])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: 1d96d476-acdb-4c44-a486-29569b3bf41e, underlay_address: fd00:1122:3344:101::b, filesystem_pool: Some(ZpoolName { id: 50a6c5cd-19db-4cbc-bda0-e365b45637b0 (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::b]:12221, external_ip: 10.151.1.167, nic: NetworkInterface { id: 3e58b382-51e7-46a7-8c87-6ca3a6547a20, kind: Service { id: 1d96d476-acdb-4c44-a486-29569b3bf41e }, name: Name("nexus-1d96d476-acdb-4c44-a486-29569b3bf41e"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 98])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: d2343105-862a-478c-b746-fd8bb78f9a96, underlay_address: fd00:1122:3344:101::c, filesystem_pool: Some(ZpoolName { id: 161cdb67-d423-452c-be58-7bca2891066e (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::c]:12221, external_ip: 10.151.1.168, nic: NetworkInterface { id: 2495ea70-4560-412b-9ebd-45a2bc27810a, kind: Service { id: d2343105-862a-478c-b746-fd8bb78f9a96 }, name: Name("nexus-d2343105-862a-478c-b746-fd8bb78f9a96"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 101])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: 391744bd-9064-4e92-ab26-8edda407dd8c, underlay_address: fd00:1122:3344:101::d, filesystem_pool: Some(ZpoolName { id: d69ccb3d-8bd7-402d-b7bd-712c30af96dc (zpool), kind: External }), zone_type: Oximeter { address: [fd00:1122:3344:101::d]:12223 } }, OmicronZoneConfig { id: 7abe137b-8d6c-4edb-b5cb-88d7f97fb593, underlay_address: fd00:1122:3344:101::e, filesystem_pool: Some(ZpoolName { id: 83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 (zpool), kind: External }), zone_type: Clickhouse { address: [fd00:1122:3344:101::e]:8123, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 (zpool), kind: External } } } }, OmicronZoneConfig { id: 6651a7a2-6c34-40eb-8170-3be50047e5a1, underlay_address: fd00:1122:3344:101::f, filesystem_pool: Some(ZpoolName { id: 1693cbb3-d027-4679-ba05-f26fe21ef04a (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::f]:17000 } }, OmicronZoneConfig { id: ff2478ea-0538-402c-8476-a8f41b89a1dd, underlay_address: fd00:1122:3344:101::10, filesystem_pool: Some(ZpoolName { id: 161cdb67-d423-452c-be58-7bca2891066e (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::10]:17000 } }, OmicronZoneConfig { id: a472c33d-180a-48f1-9999-d170d3a97f4c, underlay_address: fd00:1122:3344:101::11, filesystem_pool: Some(ZpoolName { id: 83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::11]:17000 } }, OmicronZoneConfig { id: ba6c932e-9a46-4c12-8544-9c868066cb66, underlay_address: fd00:1122:3344:101::12, filesystem_pool: Some(ZpoolName { id: 83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::12]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 (zpool), kind: External } } } }, OmicronZoneConfig { id: e1f7623b-2d1f-4b94-b59f-b88843322bd4, underlay_address: fd00:1122:3344:101::13, filesystem_pool: Some(ZpoolName { id: 1693cbb3-d027-4679-ba05-f26fe21ef04a (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::13]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 1693cbb3-d027-4679-ba05-f26fe21ef04a (zpool), kind: External } } } }, OmicronZoneConfig { id: c65dc850-b02e-4247-904b-870d82350626, underlay_address: fd00:1122:3344:101::14, filesystem_pool: Some(ZpoolName { id: 161cdb67-d423-452c-be58-7bca2891066e (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::14]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 161cdb67-d423-452c-be58-7bca2891066e (zpool), kind: External } } } }, OmicronZoneConfig { id: ad118dce-b2fc-49ed-9fb7-9a53178d3588, underlay_address: fd00:1122:3344:101::15, filesystem_pool: Some(ZpoolName { id: 3fd4f884-7e56-4c58-871f-b010d32df20b (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::15]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 3fd4f884-7e56-4c58-871f-b010d32df20b (zpool), kind: External } } } }, OmicronZoneConfig { id: 42b63708-3cd8-4e58-9c6d-786078ef5ea3, underlay_address: fd00:1122:3344:101::16, filesystem_pool: Some(ZpoolName { id: 6113cfc6-d07f-470a-963b-4465a62b65f1 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::16]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 6113cfc6-d07f-470a-963b-4465a62b65f1 (zpool), kind: External } } } }, OmicronZoneConfig { id: b7932671-3c29-4be4-828c-4ccf13631712, underlay_address: fd00:1122:3344:101::17, filesystem_pool: Some(ZpoolName { id: 9a6b9e1b-b715-4f19-b00f-4a1c2b109fdc (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::17]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9a6b9e1b-b715-4f19-b00f-4a1c2b109fdc (zpool), kind: External } } } }, OmicronZoneConfig { id: 3c79a17e-c108-4cbb-9ac6-cc88acef9672, underlay_address: fd00:1122:3344:101::18, filesystem_pool: Some(ZpoolName { id: 50a6c5cd-19db-4cbc-bda0-e365b45637b0 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::18]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 50a6c5cd-19db-4cbc-bda0-e365b45637b0 (zpool), kind: External } } } }, OmicronZoneConfig { id: 431dc29b-e656-4e8d-af66-a8f30ed60f1a, underlay_address: fd00:1122:3344:101::19, filesystem_pool: Some(ZpoolName { id: a2ff29f8-f12c-4d2f-add0-1d5faf6635ee (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::19]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a2ff29f8-f12c-4d2f-add0-1d5faf6635ee (zpool), kind: External } } } }, OmicronZoneConfig { id: f0ee96d9-e31a-49fe-8fdd-b5786e918bd9, underlay_address: fd00:1122:3344:101::1a, filesystem_pool: Some(ZpoolName { id: d69ccb3d-8bd7-402d-b7bd-712c30af96dc (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::1a]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d69ccb3d-8bd7-402d-b7bd-712c30af96dc (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:438 sled_address = [fd00:1122:3344:101::1]:12345
8252024-08-30T19:42:17.272ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:35983
8262024-08-30T19:42:17.276ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 8b642bba-10a6-428b-9af7-48d5f8622e03, dataset_name: DatasetName { pool_name: ZpoolName { id: 83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
8272024-08-30T19:42:17.276ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 8b642bba-10a6-428b-9af7-48d5f8622e03, dataset_name: DatasetName { pool_name: ZpoolName { id: 83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 (zpool), kind: External }, kind: InternalDns }, responder: ... }
8282024-08-30T19:42:17.299ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c4910af2-5760-4c5f-bb62-3b2357eab858, dataset_name: DatasetName { pool_name: ZpoolName { id: 1693cbb3-d027-4679-ba05-f26fe21ef04a (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
8292024-08-30T19:42:17.299ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: c4910af2-5760-4c5f-bb62-3b2357eab858, dataset_name: DatasetName { pool_name: ZpoolName { id: 1693cbb3-d027-4679-ba05-f26fe21ef04a (zpool), kind: External }, kind: InternalDns }, responder: ... }
8302024-08-30T19:42:17.321ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: b5d7b957-0082-4dc3-b742-d6e7150a43c2, dataset_name: DatasetName { pool_name: ZpoolName { id: 161cdb67-d423-452c-be58-7bca2891066e (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
8312024-08-30T19:42:17.321ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: b5d7b957-0082-4dc3-b742-d6e7150a43c2, dataset_name: DatasetName { pool_name: ZpoolName { id: 161cdb67-d423-452c-be58-7bca2891066e (zpool), kind: External }, kind: InternalDns }, responder: ... }
8322024-08-30T19:42:17.338ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: e5ae20e7-61c5-4df1-8e9c-36c9a79e1c3e, dataset_name: DatasetName { pool_name: ZpoolName { id: 83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
8332024-08-30T19:42:17.338ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: e5ae20e7-61c5-4df1-8e9c-36c9a79e1c3e, dataset_name: DatasetName { pool_name: ZpoolName { id: 83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8342024-08-30T19:42:17.356ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: a00a8432-4fac-47a4-9bee-ade776eac084, dataset_name: DatasetName { pool_name: ZpoolName { id: 1693cbb3-d027-4679-ba05-f26fe21ef04a (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
8352024-08-30T19:42:17.356ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: a00a8432-4fac-47a4-9bee-ade776eac084, dataset_name: DatasetName { pool_name: ZpoolName { id: 1693cbb3-d027-4679-ba05-f26fe21ef04a (zpool), kind: External }, kind: Cockroach }, responder: ... }
8362024-08-30T19:42:17.375ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 0770f0cb-550c-44e1-bda1-c4a6a97fdce7, dataset_name: DatasetName { pool_name: ZpoolName { id: 161cdb67-d423-452c-be58-7bca2891066e (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
8372024-08-30T19:42:17.375ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 0770f0cb-550c-44e1-bda1-c4a6a97fdce7, dataset_name: DatasetName { pool_name: ZpoolName { id: 161cdb67-d423-452c-be58-7bca2891066e (zpool), kind: External }, kind: Cockroach }, responder: ... }
8382024-08-30T19:42:17.393ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 47fef55b-2e70-4ed3-83d5-ed6aeeae8bc8, dataset_name: DatasetName { pool_name: ZpoolName { id: 3fd4f884-7e56-4c58-871f-b010d32df20b (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
8392024-08-30T19:42:17.393ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 47fef55b-2e70-4ed3-83d5-ed6aeeae8bc8, dataset_name: DatasetName { pool_name: ZpoolName { id: 3fd4f884-7e56-4c58-871f-b010d32df20b (zpool), kind: External }, kind: Cockroach }, responder: ... }
8402024-08-30T19:42:17.409ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 26f53738-d5c9-44ca-8ef9-8a6621900268, dataset_name: DatasetName { pool_name: ZpoolName { id: 6113cfc6-d07f-470a-963b-4465a62b65f1 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
8412024-08-30T19:42:17.409ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 26f53738-d5c9-44ca-8ef9-8a6621900268, dataset_name: DatasetName { pool_name: ZpoolName { id: 6113cfc6-d07f-470a-963b-4465a62b65f1 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8422024-08-30T19:42:17.428ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ff52340f-525a-4b88-bfc8-1cb0fe9ceba7, dataset_name: DatasetName { pool_name: ZpoolName { id: 83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 (zpool), kind: External }, kind: ExternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
8432024-08-30T19:42:17.428ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: ff52340f-525a-4b88-bfc8-1cb0fe9ceba7, dataset_name: DatasetName { pool_name: ZpoolName { id: 83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 (zpool), kind: External }, kind: ExternalDns }, responder: ... }
8442024-08-30T19:42:17.634ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: b5636b10-76ec-4aa4-8bd4-1f95780813df, dataset_name: DatasetName { pool_name: ZpoolName { id: 1693cbb3-d027-4679-ba05-f26fe21ef04a (zpool), kind: External }, kind: ExternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
8452024-08-30T19:42:17.634ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: b5636b10-76ec-4aa4-8bd4-1f95780813df, dataset_name: DatasetName { pool_name: ZpoolName { id: 1693cbb3-d027-4679-ba05-f26fe21ef04a (zpool), kind: External }, kind: ExternalDns }, responder: ... }
8462024-08-30T19:42:17.852ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 7abe137b-8d6c-4edb-b5cb-88d7f97fb593, dataset_name: DatasetName { pool_name: ZpoolName { id: 83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 (zpool), kind: External }, kind: Clickhouse }, responder: ... }) file = sled-storage/src/manager.rs:451
8472024-08-30T19:42:17.852ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 7abe137b-8d6c-4edb-b5cb-88d7f97fb593, dataset_name: DatasetName { pool_name: ZpoolName { id: 83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 (zpool), kind: External }, kind: Clickhouse }, responder: ... }
8482024-08-30T19:42:17.973ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ba6c932e-9a46-4c12-8544-9c868066cb66, dataset_name: DatasetName { pool_name: ZpoolName { id: 83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:451
8492024-08-30T19:42:17.973ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: ba6c932e-9a46-4c12-8544-9c868066cb66, dataset_name: DatasetName { pool_name: ZpoolName { id: 83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9 (zpool), kind: External }, kind: Crucible }, responder: ... }
8502024-08-30T19:42:18.108ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: e1f7623b-2d1f-4b94-b59f-b88843322bd4, dataset_name: DatasetName { pool_name: ZpoolName { id: 1693cbb3-d027-4679-ba05-f26fe21ef04a (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:451
8512024-08-30T19:42:18.108ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: e1f7623b-2d1f-4b94-b59f-b88843322bd4, dataset_name: DatasetName { pool_name: ZpoolName { id: 1693cbb3-d027-4679-ba05-f26fe21ef04a (zpool), kind: External }, kind: Crucible }, responder: ... }
8522024-08-30T19:42:18.195ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c65dc850-b02e-4247-904b-870d82350626, dataset_name: DatasetName { pool_name: ZpoolName { id: 161cdb67-d423-452c-be58-7bca2891066e (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:451
8532024-08-30T19:42:18.195ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: c65dc850-b02e-4247-904b-870d82350626, dataset_name: DatasetName { pool_name: ZpoolName { id: 161cdb67-d423-452c-be58-7bca2891066e (zpool), kind: External }, kind: Crucible }, responder: ... }
8542024-08-30T19:42:18.312ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ad118dce-b2fc-49ed-9fb7-9a53178d3588, dataset_name: DatasetName { pool_name: ZpoolName { id: 3fd4f884-7e56-4c58-871f-b010d32df20b (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:451
8552024-08-30T19:42:18.312ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: ad118dce-b2fc-49ed-9fb7-9a53178d3588, dataset_name: DatasetName { pool_name: ZpoolName { id: 3fd4f884-7e56-4c58-871f-b010d32df20b (zpool), kind: External }, kind: Crucible }, responder: ... }
8562024-08-30T19:42:18.415ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 42b63708-3cd8-4e58-9c6d-786078ef5ea3, dataset_name: DatasetName { pool_name: ZpoolName { id: 6113cfc6-d07f-470a-963b-4465a62b65f1 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:451
8572024-08-30T19:42:18.415ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 42b63708-3cd8-4e58-9c6d-786078ef5ea3, dataset_name: DatasetName { pool_name: ZpoolName { id: 6113cfc6-d07f-470a-963b-4465a62b65f1 (zpool), kind: External }, kind: Crucible }, responder: ... }
8582024-08-30T19:42:18.595ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: b7932671-3c29-4be4-828c-4ccf13631712, dataset_name: DatasetName { pool_name: ZpoolName { id: 9a6b9e1b-b715-4f19-b00f-4a1c2b109fdc (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:451
8592024-08-30T19:42:18.595ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: b7932671-3c29-4be4-828c-4ccf13631712, dataset_name: DatasetName { pool_name: ZpoolName { id: 9a6b9e1b-b715-4f19-b00f-4a1c2b109fdc (zpool), kind: External }, kind: Crucible }, responder: ... }
8602024-08-30T19:42:18.701ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 3c79a17e-c108-4cbb-9ac6-cc88acef9672, dataset_name: DatasetName { pool_name: ZpoolName { id: 50a6c5cd-19db-4cbc-bda0-e365b45637b0 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:451
8612024-08-30T19:42:18.701ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 3c79a17e-c108-4cbb-9ac6-cc88acef9672, dataset_name: DatasetName { pool_name: ZpoolName { id: 50a6c5cd-19db-4cbc-bda0-e365b45637b0 (zpool), kind: External }, kind: Crucible }, responder: ... }
8622024-08-30T19:42:18.811ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 431dc29b-e656-4e8d-af66-a8f30ed60f1a, dataset_name: DatasetName { pool_name: ZpoolName { id: a2ff29f8-f12c-4d2f-add0-1d5faf6635ee (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:451
8632024-08-30T19:42:18.811ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 431dc29b-e656-4e8d-af66-a8f30ed60f1a, dataset_name: DatasetName { pool_name: ZpoolName { id: a2ff29f8-f12c-4d2f-add0-1d5faf6635ee (zpool), kind: External }, kind: Crucible }, responder: ... }
8642024-08-30T19:42:18.908ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f0ee96d9-e31a-49fe-8fdd-b5786e918bd9, dataset_name: DatasetName { pool_name: ZpoolName { id: d69ccb3d-8bd7-402d-b7bd-712c30af96dc (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:451
8652024-08-30T19:42:18.908ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: f0ee96d9-e31a-49fe-8fdd-b5786e918bd9, dataset_name: DatasetName { pool_name: ZpoolName { id: d69ccb3d-8bd7-402d-b7bd-712c30af96dc (zpool), kind: External }, kind: Crucible }, responder: ... }
8662024-08-30T19:42:19.012ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8672024-08-30T19:42:19.012ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/fc9e5708-cd50-4950-8336-db011df04e25/config/omicron-zones.json file = common/src/ledger.rs:177
8682024-08-30T19:42:19.013ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/17afe9d4-47cb-4799-8d3d-ad69badedc16/config/omicron-zones.json file = common/src/ledger.rs:177
8692024-08-30T19:42:19.013ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8702024-08-30T19:42:19.198ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
8712024-08-30T19:42:19.198ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8722024-08-30T19:42:19.198ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8732024-08-30T19:42:19.198ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8742024-08-30T19:42:19.198ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8752024-08-30T19:42:19.198ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8762024-08-30T19:42:19.209ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
8772024-08-30T19:42:19.235ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8782024-08-30T19:42:19.235ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8792024-08-30T19:42:19.235ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8802024-08-30T19:42:19.389ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
8812024-08-30T19:42:19.390ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8822024-08-30T19:42:19.405ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
8832024-08-30T19:42:19.405ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8842024-08-30T19:42:19.405ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8852024-08-30T19:42:19.417ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
8862024-08-30T19:42:19.443ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8872024-08-30T19:42:19.443ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8882024-08-30T19:42:19.443ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8892024-08-30T19:42:19.443ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_b7932671-3c29-4be4-828c-4ccf13631712 file = illumos-utils/src/zone.rs:316
8902024-08-30T19:42:19.448ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_431dc29b-e656-4e8d-af66-a8f30ed60f1a file = illumos-utils/src/zone.rs:316
8912024-08-30T19:42:19.453ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9 file = illumos-utils/src/zone.rs:316
8922024-08-30T19:42:19.457ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_clickhouse_7abe137b-8d6c-4edb-b5cb-88d7f97fb593 file = illumos-utils/src/zone.rs:316
8932024-08-30T19:42:19.462ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_42b63708-3cd8-4e58-9c6d-786078ef5ea3 file = illumos-utils/src/zone.rs:316
8942024-08-30T19:42:19.468ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_6651a7a2-6c34-40eb-8170-3be50047e5a1 file = illumos-utils/src/zone.rs:316
8952024-08-30T19:42:19.473ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66 file = illumos-utils/src/zone.rs:316
8962024-08-30T19:42:19.477ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_ad118dce-b2fc-49ed-9fb7-9a53178d3588 file = illumos-utils/src/zone.rs:316
8972024-08-30T19:42:19.597ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_b7932671-3c29-4be4-828c-4ccf13631712 file = illumos-utils/src/zone.rs:354
8982024-08-30T19:42:19.601ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_431dc29b-e656-4e8d-af66-a8f30ed60f1a file = illumos-utils/src/zone.rs:354
8992024-08-30T19:42:19.605ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9 file = illumos-utils/src/zone.rs:354
9002024-08-30T19:42:19.609ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_clickhouse_7abe137b-8d6c-4edb-b5cb-88d7f97fb593 file = illumos-utils/src/zone.rs:354
9012024-08-30T19:42:19.613ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_42b63708-3cd8-4e58-9c6d-786078ef5ea3 file = illumos-utils/src/zone.rs:354
9022024-08-30T19:42:19.618ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_6651a7a2-6c34-40eb-8170-3be50047e5a1 file = illumos-utils/src/zone.rs:354
9032024-08-30T19:42:19.622ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66 file = illumos-utils/src/zone.rs:354
9042024-08-30T19:42:19.626ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_ad118dce-b2fc-49ed-9fb7-9a53178d3588 file = illumos-utils/src/zone.rs:354
9052024-08-30T19:42:19.632ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_oximeter_391744bd-9064-4e92-ab26-8edda407dd8c file = illumos-utils/src/zone.rs:316
9062024-08-30T19:42:19.638ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_c65dc850-b02e-4247-904b-870d82350626 file = illumos-utils/src/zone.rs:316
9072024-08-30T19:42:19.642ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_a472c33d-180a-48f1-9999-d170d3a97f4c file = illumos-utils/src/zone.rs:316
9082024-08-30T19:42:19.688ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
9092024-08-30T19:42:19.688ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_ff2478ea-0538-402c-8476-a8f41b89a1dd file = illumos-utils/src/zone.rs:316
9102024-08-30T19:42:19.694ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_e1f7623b-2d1f-4b94-b59f-b88843322bd4 file = illumos-utils/src/zone.rs:316
9112024-08-30T19:42:19.700ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_3c79a17e-c108-4cbb-9ac6-cc88acef9672 file = illumos-utils/src/zone.rs:316
9122024-08-30T19:42:19.745ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
9132024-08-30T19:42:19.746ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_oximeter_391744bd-9064-4e92-ab26-8edda407dd8c file = illumos-utils/src/zone.rs:354
9142024-08-30T19:42:19.751ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_c65dc850-b02e-4247-904b-870d82350626 file = illumos-utils/src/zone.rs:354
9152024-08-30T19:42:19.755ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_a472c33d-180a-48f1-9999-d170d3a97f4c file = illumos-utils/src/zone.rs:354
9162024-08-30T19:42:19.760ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_ff2478ea-0538-402c-8476-a8f41b89a1dd file = illumos-utils/src/zone.rs:354
9172024-08-30T19:42:19.765ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_e1f7623b-2d1f-4b94-b59f-b88843322bd4 file = illumos-utils/src/zone.rs:354
9182024-08-30T19:42:19.771ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_3c79a17e-c108-4cbb-9ac6-cc88acef9672 file = illumos-utils/src/zone.rs:354
9192024-08-30T19:42:19.810ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
9202024-08-30T19:42:19.854ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
9212024-08-30T19:42:19.861ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
9222024-08-30T19:42:19.861ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
9232024-08-30T19:42:19.913ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte1", ip: 172.30.2.5, mac: MacAddr6([168, 64, 37, 255, 158, 211]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
9242024-08-30T19:42:19.913ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1234 zone_type = nexus
9252024-08-30T19:42:19.915ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
9262024-08-30T19:42:19.915ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
9272024-08-30T19:42:19.958ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte2", ip: 172.30.2.6, mac: MacAddr6([168, 64, 37, 255, 225, 98]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
9282024-08-30T19:42:19.958ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1234 zone_type = nexus
9292024-08-30T19:42:20.010ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
9302024-08-30T19:42:20.013ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
9312024-08-30T19:42:20.013ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
9322024-08-30T19:42:20.058ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte3", ip: 172.30.1.6, mac: MacAddr6([168, 64, 37, 255, 151, 107]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.1.1 } }) }
9332024-08-30T19:42:20.058ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1234 zone_type = external_dns
9342024-08-30T19:42:20.059ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
9352024-08-30T19:42:20.059ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
9362024-08-30T19:42:20.106ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte4", ip: 172.30.2.7, mac: MacAddr6([168, 64, 37, 255, 203, 101]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
9372024-08-30T19:42:20.106ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1234 zone_type = nexus
9382024-08-30T19:42:20.109ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
9392024-08-30T19:42:20.109ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
9402024-08-30T19:42:20.157ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte5", ip: 172.30.1.5, mac: MacAddr6([168, 64, 37, 255, 214, 231]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.1.1 } }) }
9412024-08-30T19:42:20.157ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1234 zone_type = external_dns
9422024-08-30T19:42:20.157ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.1.167 file = clients/dpd-client/src/lib.rs:83
9432024-08-30T19:42:20.157ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.1.167 file = clients/dpd-client/src/lib.rs:90
9442024-08-30T19:42:20.157ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.1.165 file = clients/dpd-client/src/lib.rs:83
9452024-08-30T19:42:20.157ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.1.165 file = clients/dpd-client/src/lib.rs:90
9462024-08-30T19:42:20.157ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.1.168 file = clients/dpd-client/src/lib.rs:83
9472024-08-30T19:42:20.157ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.1.168 file = clients/dpd-client/src/lib.rs:90
9482024-08-30T19:42:20.157ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.1.166 file = clients/dpd-client/src/lib.rs:83
9492024-08-30T19:42:20.157ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.1.166 file = clients/dpd-client/src/lib.rs:90
9502024-08-30T19:42:20.177ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.1.164 file = clients/dpd-client/src/lib.rs:83
9512024-08-30T19:42:20.177ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.1.164 file = clients/dpd-client/src/lib.rs:90
9522024-08-30T19:42:20.177ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.1.167 file = clients/dpd-client/src/lib.rs:118
9532024-08-30T19:42:20.180ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.1.165 file = clients/dpd-client/src/lib.rs:118
9542024-08-30T19:42:20.183ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
9552024-08-30T19:42:20.183ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.1.168 file = clients/dpd-client/src/lib.rs:118
9562024-08-30T19:42:20.183ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
9572024-08-30T19:42:20.185ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.1.166 file = clients/dpd-client/src/lib.rs:118
9582024-08-30T19:42:20.215ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.1.164 file = clients/dpd-client/src/lib.rs:118
9592024-08-30T19:42:20.215ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
9602024-08-30T19:42:20.215ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
9612024-08-30T19:42:20.232ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
9622024-08-30T19:42:20.487ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_1d96d476-acdb-4c44-a486-29569b3bf41e file = illumos-utils/src/zone.rs:316
9632024-08-30T19:42:20.511ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_external_dns_b5636b10-76ec-4aa4-8bd4-1f95780813df file = illumos-utils/src/zone.rs:316
9642024-08-30T19:42:20.518ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_d2343105-862a-478c-b746-fd8bb78f9a96 file = illumos-utils/src/zone.rs:316
9652024-08-30T19:42:20.529ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_ba4aef74-7b33-4ded-a46a-dc2534b0db9d file = illumos-utils/src/zone.rs:316
9662024-08-30T19:42:20.542ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_external_dns_ff52340f-525a-4b88-bfc8-1cb0fe9ceba7 file = illumos-utils/src/zone.rs:316
9672024-08-30T19:42:20.554ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_1d96d476-acdb-4c44-a486-29569b3bf41e file = illumos-utils/src/zone.rs:354
9682024-08-30T19:42:20.568ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_external_dns_b5636b10-76ec-4aa4-8bd4-1f95780813df file = illumos-utils/src/zone.rs:354
9692024-08-30T19:42:20.582ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_d2343105-862a-478c-b746-fd8bb78f9a96 file = illumos-utils/src/zone.rs:354
9702024-08-30T19:42:20.595ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_ba4aef74-7b33-4ded-a46a-dc2534b0db9d file = illumos-utils/src/zone.rs:354
9712024-08-30T19:42:20.607ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_external_dns_ff52340f-525a-4b88-bfc8-1cb0fe9ceba7 file = illumos-utils/src/zone.rs:354
9722024-08-30T19:42:23.137ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_6651a7a2-6c34-40eb-8170-3be50047e5a1: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService15'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::f'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::f'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9732024-08-30T19:42:23.137ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_a472c33d-180a-48f1-9999-d170d3a97f4c: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService20'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::11'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::11'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9742024-08-30T19:42:23.137ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_6651a7a2-6c34-40eb-8170-3be50047e5a1 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_6651a7a2-6c34-40eb-8170-3be50047e5a1
9752024-08-30T19:42:23.147ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_a472c33d-180a-48f1-9999-d170d3a97f4c zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_a472c33d-180a-48f1-9999-d170d3a97f4c
9762024-08-30T19:42:23.304ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_431dc29b-e656-4e8d-af66-a8f30ed60f1a: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService11'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::19'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_a2ff29f8-f12c-4d2f-add0-1d5faf6635ee/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::19'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='1ced94b9-d3f7-4f66-a0b9-0dae10130a81'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9772024-08-30T19:42:23.305ZINFOSledAgent (ServiceManager): Booting oxz_crucible_431dc29b-e656-4e8d-af66-a8f30ed60f1a zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_431dc29b-e656-4e8d-af66-a8f30ed60f1a
9782024-08-30T19:42:23.317ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService12'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::1a'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_d69ccb3d-8bd7-402d-b7bd-712c30af96dc/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::1a'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='d0a327b2-5d6b-4d6d-86d1-47e1f32c4ec8'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9792024-08-30T19:42:23.317ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_42b63708-3cd8-4e58-9c6d-786078ef5ea3: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService14'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::16'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_6113cfc6-d07f-470a-963b-4465a62b65f1/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::16'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='e806d56f-555a-4dff-8ec5-ca15eafaaeef'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9802024-08-30T19:42:23.320ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService16'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::12'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::12'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='2522be27-482e-4103-b734-03442cb1f5ca'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9812024-08-30T19:42:23.320ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_ad118dce-b2fc-49ed-9fb7-9a53178d3588: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService17'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::15'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_3fd4f884-7e56-4c58-871f-b010d32df20b/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::15'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='1ad10992-8471-40de-9738-185a761135bb'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9822024-08-30T19:42:23.321ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_3c79a17e-c108-4cbb-9ac6-cc88acef9672: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService23'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::18'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_50a6c5cd-19db-4cbc-bda0-e365b45637b0/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::18'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='90c6cfaf-39af-4541-8031-407b189f1b62'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9832024-08-30T19:42:23.321ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_b7932671-3c29-4be4-828c-4ccf13631712: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService10'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::17'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_9a6b9e1b-b715-4f19-b00f-4a1c2b109fdc/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::17'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='2e922ce0-b0fd-46ae-b5f6-867bcd8e0c41'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9842024-08-30T19:42:23.321ZINFOSledAgent (ServiceManager): Booting oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9
9852024-08-30T19:42:23.333ZINFOSledAgent (ServiceManager): Booting oxz_crucible_42b63708-3cd8-4e58-9c6d-786078ef5ea3 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_42b63708-3cd8-4e58-9c6d-786078ef5ea3
9862024-08-30T19:42:23.343ZINFOSledAgent (ServiceManager): Booting oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66
9872024-08-30T19:42:23.360ZINFOSledAgent (ServiceManager): Booting oxz_crucible_ad118dce-b2fc-49ed-9fb7-9a53178d3588 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_ad118dce-b2fc-49ed-9fb7-9a53178d3588
9882024-08-30T19:42:23.380ZINFOSledAgent (ServiceManager): Booting oxz_crucible_3c79a17e-c108-4cbb-9ac6-cc88acef9672 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_3c79a17e-c108-4cbb-9ac6-cc88acef9672
9892024-08-30T19:42:23.398ZINFOSledAgent (ServiceManager): Booting oxz_crucible_b7932671-3c29-4be4-828c-4ccf13631712 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_b7932671-3c29-4be4-828c-4ccf13631712
9902024-08-30T19:42:23.594ZINFOSledAgent (ServiceManager): Profile for oxz_oximeter_391744bd-9064-4e92-ab26-8edda407dd8c: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService18'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::d'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/oximeter"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="id" value='391744bd-9064-4e92-ab26-8edda407dd8c'/> <propval type="astring" name="address" value='[fd00:1122:3344:101::d]:12223'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9912024-08-30T19:42:23.597ZINFOSledAgent (ServiceManager): Booting oxz_oximeter_391744bd-9064-4e92-ab26-8edda407dd8c zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_oximeter_391744bd-9064-4e92-ab26-8edda407dd8c
9922024-08-30T19:42:23.782ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ad118dce-b2fc-49ed-9fb7-9a53178d3588") failed: Property not found. retry in 69.833173ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ad118dce-b2fc-49ed-9fb7-9a53178d3588
9932024-08-30T19:42:23.793ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b7932671-3c29-4be4-828c-4ccf13631712") failed: Property not found. retry in 42.424396ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b7932671-3c29-4be4-828c-4ccf13631712
9942024-08-30T19:42:23.802ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_431dc29b-e656-4e8d-af66-a8f30ed60f1a") failed: Property not found. retry in 46.161471ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_431dc29b-e656-4e8d-af66-a8f30ed60f1a
9952024-08-30T19:42:23.812ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9") failed: Property not found. retry in 46.154245ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9
9962024-08-30T19:42:23.820ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_42b63708-3cd8-4e58-9c6d-786078ef5ea3") failed: Property not found. retry in 64.870525ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_42b63708-3cd8-4e58-9c6d-786078ef5ea3
9972024-08-30T19:42:23.829ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66") failed: Property not found. retry in 25.396333ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66
9982024-08-30T19:42:23.837ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_a472c33d-180a-48f1-9999-d170d3a97f4c") failed: Property not found. retry in 43.972861ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_a472c33d-180a-48f1-9999-d170d3a97f4c
9992024-08-30T19:42:23.846ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c79a17e-c108-4cbb-9ac6-cc88acef9672") failed: Property not found. retry in 57.293596ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c79a17e-c108-4cbb-9ac6-cc88acef9672
10002024-08-30T19:42:23.901ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b7932671-3c29-4be4-828c-4ccf13631712") failed: Property not found. retry in 82.255709ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b7932671-3c29-4be4-828c-4ccf13631712
10012024-08-30T19:42:23.967ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_431dc29b-e656-4e8d-af66-a8f30ed60f1a") failed: Property not found. retry in 100.831054ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_431dc29b-e656-4e8d-af66-a8f30ed60f1a
10022024-08-30T19:42:24.197ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9") failed: Property not found. retry in 108.188774ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9
10032024-08-30T19:42:24.304ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_42b63708-3cd8-4e58-9c6d-786078ef5ea3") failed: Property not found. retry in 62.775346ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_42b63708-3cd8-4e58-9c6d-786078ef5ea3
10042024-08-30T19:42:24.641ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66") failed: Property not found. retry in 58.890674ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66
10052024-08-30T19:42:24.837ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ad118dce-b2fc-49ed-9fb7-9a53178d3588") failed: Property not found. retry in 139.076123ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ad118dce-b2fc-49ed-9fb7-9a53178d3588
10062024-08-30T19:42:24.961ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_391744bd-9064-4e92-ab26-8edda407dd8c") failed: Property not found. retry in 72.407966ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_391744bd-9064-4e92-ab26-8edda407dd8c
10072024-08-30T19:42:25.003ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_a472c33d-180a-48f1-9999-d170d3a97f4c") failed: Property not found. retry in 149.604908ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_a472c33d-180a-48f1-9999-d170d3a97f4c
10082024-08-30T19:42:25.039ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b7932671-3c29-4be4-828c-4ccf13631712") failed: Property not found. retry in 188.604901ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b7932671-3c29-4be4-828c-4ccf13631712
10092024-08-30T19:42:25.266ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_431dc29b-e656-4e8d-af66-a8f30ed60f1a") failed: Property not found. retry in 159.674082ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_431dc29b-e656-4e8d-af66-a8f30ed60f1a
10102024-08-30T19:42:25.320ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9") failed: Property not found. retry in 294.371227ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9
10112024-08-30T19:42:25.358ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_42b63708-3cd8-4e58-9c6d-786078ef5ea3") failed: Property not found. retry in 103.797214ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_42b63708-3cd8-4e58-9c6d-786078ef5ea3
10122024-08-30T19:42:25.403ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66") failed: Property not found. retry in 128.825761ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66
10132024-08-30T19:42:25.422ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ad118dce-b2fc-49ed-9fb7-9a53178d3588") failed: Property not found. retry in 257.287632ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ad118dce-b2fc-49ed-9fb7-9a53178d3588
10142024-08-30T19:42:25.439ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_391744bd-9064-4e92-ab26-8edda407dd8c") failed: Property not found. retry in 106.76496ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_391744bd-9064-4e92-ab26-8edda407dd8c
10152024-08-30T19:42:25.452ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_a472c33d-180a-48f1-9999-d170d3a97f4c") failed: Property not found. retry in 277.907089ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_a472c33d-180a-48f1-9999-d170d3a97f4c
10162024-08-30T19:42:25.470ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c79a17e-c108-4cbb-9ac6-cc88acef9672") failed: Property not found. retry in 72.858347ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c79a17e-c108-4cbb-9ac6-cc88acef9672
10172024-08-30T19:42:25.486ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b7932671-3c29-4be4-828c-4ccf13631712") failed: Property not found. retry in 276.877258ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b7932671-3c29-4be4-828c-4ccf13631712
10182024-08-30T19:42:25.499ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_431dc29b-e656-4e8d-af66-a8f30ed60f1a") failed: Property not found. retry in 547.921384ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_431dc29b-e656-4e8d-af66-a8f30ed60f1a
10192024-08-30T19:42:25.513ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_42b63708-3cd8-4e58-9c6d-786078ef5ea3") failed: Property not found. retry in 398.440846ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_42b63708-3cd8-4e58-9c6d-786078ef5ea3
10202024-08-30T19:42:25.533ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_ff2478ea-0538-402c-8476-a8f41b89a1dd: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService21'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::10'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::10'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10212024-08-30T19:42:25.547ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66") failed: Property not found. retry in 426.051114ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66
10222024-08-30T19:42:25.547ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_ff2478ea-0538-402c-8476-a8f41b89a1dd zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_ff2478ea-0538-402c-8476-a8f41b89a1dd
10232024-08-30T19:42:25.567ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_391744bd-9064-4e92-ab26-8edda407dd8c") failed: Property not found. retry in 147.125674ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_391744bd-9064-4e92-ab26-8edda407dd8c
10242024-08-30T19:42:25.580ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c79a17e-c108-4cbb-9ac6-cc88acef9672") failed: Property not found. retry in 172.540353ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c79a17e-c108-4cbb-9ac6-cc88acef9672
10252024-08-30T19:42:25.627ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9") failed: Property not found. retry in 424.679009ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9
10262024-08-30T19:42:25.676ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_c65dc850-b02e-4247-904b-870d82350626: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService19'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::14'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_161cdb67-d423-452c-be58-7bca2891066e/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::14'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='543b8137-9ddb-424d-a507-fd962e06e273'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10272024-08-30T19:42:25.676ZINFOSledAgent (ServiceManager): Booting oxz_crucible_c65dc850-b02e-4247-904b-870d82350626 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_c65dc850-b02e-4247-904b-870d82350626
10282024-08-30T19:42:25.684ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_e1f7623b-2d1f-4b94-b59f-b88843322bd4: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService22'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::13'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_1693cbb3-d027-4679-ba05-f26fe21ef04a/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::13'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='c03c82be-3a61-4521-9674-dfb33f290b0f'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10292024-08-30T19:42:25.697ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ad118dce-b2fc-49ed-9fb7-9a53178d3588") failed: Property not found. retry in 509.453559ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ad118dce-b2fc-49ed-9fb7-9a53178d3588
10302024-08-30T19:42:25.697ZINFOSledAgent (ServiceManager): Booting oxz_crucible_e1f7623b-2d1f-4b94-b59f-b88843322bd4 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_e1f7623b-2d1f-4b94-b59f-b88843322bd4
10312024-08-30T19:42:25.725ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_391744bd-9064-4e92-ab26-8edda407dd8c") failed: Property not found. retry in 254.298924ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_391744bd-9064-4e92-ab26-8edda407dd8c
10322024-08-30T19:42:25.741ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_a472c33d-180a-48f1-9999-d170d3a97f4c") failed: Property not found. retry in 214.085588ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_a472c33d-180a-48f1-9999-d170d3a97f4c
10332024-08-30T19:42:25.767ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c79a17e-c108-4cbb-9ac6-cc88acef9672") failed: Property not found. retry in 501.62728ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c79a17e-c108-4cbb-9ac6-cc88acef9672
10342024-08-30T19:42:25.779ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b7932671-3c29-4be4-828c-4ccf13631712") failed: Property not found. retry in 1.101939612s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b7932671-3c29-4be4-828c-4ccf13631712
10352024-08-30T19:42:25.791ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6651a7a2-6c34-40eb-8170-3be50047e5a1") failed: Property not found. retry in 69.392394ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6651a7a2-6c34-40eb-8170-3be50047e5a1
10362024-08-30T19:42:25.807ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_ff2478ea-0538-402c-8476-a8f41b89a1dd") failed: Property not found. retry in 28.269016ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_ff2478ea-0538-402c-8476-a8f41b89a1dd
10372024-08-30T19:42:25.830ZINFOSledAgent (ServiceManager): Profile for oxz_external_dns_b5636b10-76ec-4aa4-8bd4-1f95780813df: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService25'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::9'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte3'/> <propval type="astring" name="gateway" value='172.30.1.1'/> <propval type="astring" name="ip" value='172.30.1.6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/external_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::9]:5353'/> <propval type="astring" name="dns_address" value='172.30.1.6:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10382024-08-30T19:42:25.830ZINFOSledAgent (ServiceManager): Booting oxz_external_dns_b5636b10-76ec-4aa4-8bd4-1f95780813df zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_external_dns_b5636b10-76ec-4aa4-8bd4-1f95780813df
10392024-08-30T19:42:25.847ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_ff2478ea-0538-402c-8476-a8f41b89a1dd") failed: Property not found. retry in 110.472239ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_ff2478ea-0538-402c-8476-a8f41b89a1dd
10402024-08-30T19:42:25.871ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6651a7a2-6c34-40eb-8170-3be50047e5a1") failed: Property not found. retry in 67.732721ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6651a7a2-6c34-40eb-8170-3be50047e5a1
10412024-08-30T19:42:25.925ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e1f7623b-2d1f-4b94-b59f-b88843322bd4") failed: Property not found. retry in 25.699082ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e1f7623b-2d1f-4b94-b59f-b88843322bd4
10422024-08-30T19:42:25.936ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_42b63708-3cd8-4e58-9c6d-786078ef5ea3") failed: Property not found. retry in 433.731331ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_42b63708-3cd8-4e58-9c6d-786078ef5ea3
10432024-08-30T19:42:25.980ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c65dc850-b02e-4247-904b-870d82350626") failed: Property not found. retry in 47.257954ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c65dc850-b02e-4247-904b-870d82350626
10442024-08-30T19:42:26.015ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6651a7a2-6c34-40eb-8170-3be50047e5a1") failed: Property not found. retry in 101.399495ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6651a7a2-6c34-40eb-8170-3be50047e5a1
10452024-08-30T19:42:26.104ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66") failed: Property not found. retry in 574.265789ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66
10462024-08-30T19:42:26.117ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_391744bd-9064-4e92-ab26-8edda407dd8c") failed: Property not found. retry in 896.688897ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_391744bd-9064-4e92-ab26-8edda407dd8c
10472024-08-30T19:42:26.129ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_a472c33d-180a-48f1-9999-d170d3a97f4c") failed: Property not found. retry in 483.783118ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_a472c33d-180a-48f1-9999-d170d3a97f4c
10482024-08-30T19:42:26.143ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_ff2478ea-0538-402c-8476-a8f41b89a1dd") failed: Property not found. retry in 132.04647ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_ff2478ea-0538-402c-8476-a8f41b89a1dd
10492024-08-30T19:42:26.155ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e1f7623b-2d1f-4b94-b59f-b88843322bd4") failed: Property not found. retry in 119.799969ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e1f7623b-2d1f-4b94-b59f-b88843322bd4
10502024-08-30T19:42:26.169ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_431dc29b-e656-4e8d-af66-a8f30ed60f1a") failed: Property not found. retry in 430.989729ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_431dc29b-e656-4e8d-af66-a8f30ed60f1a
10512024-08-30T19:42:26.183ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9") failed: Property not found. retry in 401.799445ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9
10522024-08-30T19:42:26.196ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_b5636b10-76ec-4aa4-8bd4-1f95780813df") failed: Property not found. retry in 38.323723ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_b5636b10-76ec-4aa4-8bd4-1f95780813df
10532024-08-30T19:42:26.210ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6651a7a2-6c34-40eb-8170-3be50047e5a1") failed: Property not found. retry in 595.778485ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6651a7a2-6c34-40eb-8170-3be50047e5a1
10542024-08-30T19:42:26.223ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ad118dce-b2fc-49ed-9fb7-9a53178d3588") failed: Property not found. retry in 730.8333ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ad118dce-b2fc-49ed-9fb7-9a53178d3588
10552024-08-30T19:42:26.232ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c65dc850-b02e-4247-904b-870d82350626") failed: Property not found. retry in 148.965082ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c65dc850-b02e-4247-904b-870d82350626
10562024-08-30T19:42:26.245ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_b5636b10-76ec-4aa4-8bd4-1f95780813df") failed: Property not found. retry in 125.825966ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_b5636b10-76ec-4aa4-8bd4-1f95780813df
10572024-08-30T19:42:26.285ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c79a17e-c108-4cbb-9ac6-cc88acef9672") failed: Property not found. retry in 782.099372ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c79a17e-c108-4cbb-9ac6-cc88acef9672
10582024-08-30T19:42:26.300ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_ff2478ea-0538-402c-8476-a8f41b89a1dd") failed: Property not found. retry in 342.664095ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_ff2478ea-0538-402c-8476-a8f41b89a1dd
10592024-08-30T19:42:26.318ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e1f7623b-2d1f-4b94-b59f-b88843322bd4") failed: Property not found. retry in 138.792452ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e1f7623b-2d1f-4b94-b59f-b88843322bd4
10602024-08-30T19:42:26.387ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_42b63708-3cd8-4e58-9c6d-786078ef5ea3") failed: Property not found. retry in 715.622433ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_42b63708-3cd8-4e58-9c6d-786078ef5ea3
10612024-08-30T19:42:26.401ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_b5636b10-76ec-4aa4-8bd4-1f95780813df") failed: Property not found. retry in 281.270226ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_b5636b10-76ec-4aa4-8bd4-1f95780813df
10622024-08-30T19:42:26.420ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c65dc850-b02e-4247-904b-870d82350626") failed: Property not found. retry in 186.605002ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c65dc850-b02e-4247-904b-870d82350626
10632024-08-30T19:42:26.473ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e1f7623b-2d1f-4b94-b59f-b88843322bd4") failed: Property not found. retry in 294.803525ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e1f7623b-2d1f-4b94-b59f-b88843322bd4
10642024-08-30T19:42:26.605ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9") failed: Property not found. retry in 716.529608ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9
10652024-08-30T19:42:26.623ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_431dc29b-e656-4e8d-af66-a8f30ed60f1a") failed: Property not found. retry in 1.164226389s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_431dc29b-e656-4e8d-af66-a8f30ed60f1a
10662024-08-30T19:42:26.645ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c65dc850-b02e-4247-904b-870d82350626") failed: Property not found. retry in 498.100976ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c65dc850-b02e-4247-904b-870d82350626
10672024-08-30T19:42:26.657ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_a472c33d-180a-48f1-9999-d170d3a97f4c") failed: Property not found. retry in 1.322026137s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_a472c33d-180a-48f1-9999-d170d3a97f4c
10682024-08-30T19:42:26.690ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_ff2478ea-0538-402c-8476-a8f41b89a1dd") failed: Property not found. retry in 833.600041ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_ff2478ea-0538-402c-8476-a8f41b89a1dd
10692024-08-30T19:42:26.708ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_b5636b10-76ec-4aa4-8bd4-1f95780813df") failed: Property not found. retry in 371.633031ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_b5636b10-76ec-4aa4-8bd4-1f95780813df
10702024-08-30T19:42:26.720ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66") failed: Property not found. retry in 1.319306375s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66
10712024-08-30T19:42:26.721ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_ba4aef74-7b33-4ded-a46a-dc2534b0db9d: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService27'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::a'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte1'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10722024-08-30T19:42:26.721ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_1d96d476-acdb-4c44-a486-29569b3bf41e: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService24'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::b'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte2'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10732024-08-30T19:42:26.721ZINFOSledAgent (ServiceManager): Booting oxz_nexus_ba4aef74-7b33-4ded-a46a-dc2534b0db9d zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_ba4aef74-7b33-4ded-a46a-dc2534b0db9d
10742024-08-30T19:42:26.731ZINFOSledAgent (ServiceManager): Booting oxz_nexus_1d96d476-acdb-4c44-a486-29569b3bf41e zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_1d96d476-acdb-4c44-a486-29569b3bf41e
10752024-08-30T19:42:26.785ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e1f7623b-2d1f-4b94-b59f-b88843322bd4") failed: Property not found. retry in 870.063731ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e1f7623b-2d1f-4b94-b59f-b88843322bd4
10762024-08-30T19:42:26.823ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6651a7a2-6c34-40eb-8170-3be50047e5a1") failed: Property not found. retry in 717.11836ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6651a7a2-6c34-40eb-8170-3be50047e5a1
10772024-08-30T19:42:26.825ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_d2343105-862a-478c-b746-fd8bb78f9a96: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService26'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::c'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte4'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.7'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10782024-08-30T19:42:26.826ZINFOSledAgent (ServiceManager): Booting oxz_nexus_d2343105-862a-478c-b746-fd8bb78f9a96 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_d2343105-862a-478c-b746-fd8bb78f9a96
10792024-08-30T19:42:26.898ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b7932671-3c29-4be4-828c-4ccf13631712") failed: Property not found. retry in 1.28056745s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b7932671-3c29-4be4-828c-4ccf13631712
10802024-08-30T19:42:27.039ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ad118dce-b2fc-49ed-9fb7-9a53178d3588") failed: Property not found. retry in 1.406488802s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ad118dce-b2fc-49ed-9fb7-9a53178d3588
10812024-08-30T19:42:27.076ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_391744bd-9064-4e92-ab26-8edda407dd8c") failed: Property not found. retry in 1.487746832s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_391744bd-9064-4e92-ab26-8edda407dd8c
10822024-08-30T19:42:27.121ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c79a17e-c108-4cbb-9ac6-cc88acef9672") failed: Property not found. retry in 643.19629ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c79a17e-c108-4cbb-9ac6-cc88acef9672
10832024-08-30T19:42:27.143ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_42b63708-3cd8-4e58-9c6d-786078ef5ea3") failed: Property not found. retry in 868.290295ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_42b63708-3cd8-4e58-9c6d-786078ef5ea3
10842024-08-30T19:42:27.158ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_b5636b10-76ec-4aa4-8bd4-1f95780813df") failed: Property not found. retry in 1.105660684s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_b5636b10-76ec-4aa4-8bd4-1f95780813df
10852024-08-30T19:42:27.172ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ba4aef74-7b33-4ded-a46a-dc2534b0db9d") failed: Property not found. retry in 58.124105ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ba4aef74-7b33-4ded-a46a-dc2534b0db9d
10862024-08-30T19:42:27.186ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1d96d476-acdb-4c44-a486-29569b3bf41e") failed: Property not found. retry in 33.071641ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1d96d476-acdb-4c44-a486-29569b3bf41e
10872024-08-30T19:42:27.202ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c65dc850-b02e-4247-904b-870d82350626") failed: Property not found. retry in 490.46911ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c65dc850-b02e-4247-904b-870d82350626
10882024-08-30T19:42:27.233ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1d96d476-acdb-4c44-a486-29569b3bf41e") failed: Property not found. retry in 106.622442ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1d96d476-acdb-4c44-a486-29569b3bf41e
10892024-08-30T19:42:27.250ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ba4aef74-7b33-4ded-a46a-dc2534b0db9d") failed: Property not found. retry in 50.335417ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ba4aef74-7b33-4ded-a46a-dc2534b0db9d
10902024-08-30T19:42:27.264ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d2343105-862a-478c-b746-fd8bb78f9a96") failed: Property not found. retry in 52.008738ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d2343105-862a-478c-b746-fd8bb78f9a96
10912024-08-30T19:42:27.318ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ba4aef74-7b33-4ded-a46a-dc2534b0db9d") failed: Property not found. retry in 188.453098ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ba4aef74-7b33-4ded-a46a-dc2534b0db9d
10922024-08-30T19:42:27.329ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d2343105-862a-478c-b746-fd8bb78f9a96") failed: Property not found. retry in 112.748471ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d2343105-862a-478c-b746-fd8bb78f9a96
10932024-08-30T19:42:27.343ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9") failed: Property not found. retry in 770.364201ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9
10942024-08-30T19:42:27.357ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1d96d476-acdb-4c44-a486-29569b3bf41e") failed: Property not found. retry in 225.463122ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1d96d476-acdb-4c44-a486-29569b3bf41e
10952024-08-30T19:42:27.456ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d2343105-862a-478c-b746-fd8bb78f9a96") failed: Property not found. retry in 109.829184ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d2343105-862a-478c-b746-fd8bb78f9a96
10962024-08-30T19:42:27.524ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ba4aef74-7b33-4ded-a46a-dc2534b0db9d") failed: Property not found. retry in 391.310601ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ba4aef74-7b33-4ded-a46a-dc2534b0db9d
10972024-08-30T19:42:27.539ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_ff2478ea-0538-402c-8476-a8f41b89a1dd") failed: Property not found. retry in 1.117697419s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_ff2478ea-0538-402c-8476-a8f41b89a1dd
10982024-08-30T19:42:27.553ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6651a7a2-6c34-40eb-8170-3be50047e5a1") failed: Property not found. retry in 585.541647ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6651a7a2-6c34-40eb-8170-3be50047e5a1
10992024-08-30T19:42:27.580ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d2343105-862a-478c-b746-fd8bb78f9a96") failed: Property not found. retry in 535.357064ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d2343105-862a-478c-b746-fd8bb78f9a96
11002024-08-30T19:42:27.598ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1d96d476-acdb-4c44-a486-29569b3bf41e") failed: Property not found. retry in 429.20926ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1d96d476-acdb-4c44-a486-29569b3bf41e
11012024-08-30T19:42:27.670ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e1f7623b-2d1f-4b94-b59f-b88843322bd4") failed: Property not found. retry in 1.293962492s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e1f7623b-2d1f-4b94-b59f-b88843322bd4
11022024-08-30T19:42:27.709ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c65dc850-b02e-4247-904b-870d82350626") failed: Property not found. retry in 1.413510132s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c65dc850-b02e-4247-904b-870d82350626
11032024-08-30T19:42:27.780ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c79a17e-c108-4cbb-9ac6-cc88acef9672") failed: Property not found. retry in 1.162360949s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c79a17e-c108-4cbb-9ac6-cc88acef9672
11042024-08-30T19:42:27.799ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_431dc29b-e656-4e8d-af66-a8f30ed60f1a") failed: Property not found. retry in 853.163705ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_431dc29b-e656-4e8d-af66-a8f30ed60f1a
11052024-08-30T19:42:27.929ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ba4aef74-7b33-4ded-a46a-dc2534b0db9d") failed: Property not found. retry in 999.189213ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ba4aef74-7b33-4ded-a46a-dc2534b0db9d
11062024-08-30T19:42:27.990ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_a472c33d-180a-48f1-9999-d170d3a97f4c") failed: Property not found. retry in 1.443013546s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_a472c33d-180a-48f1-9999-d170d3a97f4c
11072024-08-30T19:42:28.022ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_42b63708-3cd8-4e58-9c6d-786078ef5ea3") failed: Property not found. retry in 745.664842ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_42b63708-3cd8-4e58-9c6d-786078ef5ea3
11082024-08-30T19:42:28.040ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1d96d476-acdb-4c44-a486-29569b3bf41e") failed: Property not found. retry in 1.017235402s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1d96d476-acdb-4c44-a486-29569b3bf41e
11092024-08-30T19:42:28.049ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66") failed: Property not found. retry in 563.52588ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66
11102024-08-30T19:42:28.124ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9") failed: Property not found. retry in 520.35837ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9
11112024-08-30T19:42:28.135ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d2343105-862a-478c-b746-fd8bb78f9a96") failed: Property not found. retry in 916.088165ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d2343105-862a-478c-b746-fd8bb78f9a96
11122024-08-30T19:42:28.150ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6651a7a2-6c34-40eb-8170-3be50047e5a1") failed: Property not found. retry in 1.151500623s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6651a7a2-6c34-40eb-8170-3be50047e5a1
11132024-08-30T19:42:28.191ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b7932671-3c29-4be4-828c-4ccf13631712") failed: Property not found. retry in 1.29814072s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b7932671-3c29-4be4-828c-4ccf13631712
11142024-08-30T19:42:28.278ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_b5636b10-76ec-4aa4-8bd4-1f95780813df") failed: Property not found. retry in 869.602539ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_b5636b10-76ec-4aa4-8bd4-1f95780813df
11152024-08-30T19:42:28.458ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ad118dce-b2fc-49ed-9fb7-9a53178d3588") failed: Property not found. retry in 952.167535ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ad118dce-b2fc-49ed-9fb7-9a53178d3588
11162024-08-30T19:42:28.578ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_391744bd-9064-4e92-ab26-8edda407dd8c") failed: Property not found. retry in 678.584156ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_391744bd-9064-4e92-ab26-8edda407dd8c
11172024-08-30T19:42:28.625ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66") failed: Property not found. retry in 1.29169088s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66
11182024-08-30T19:42:28.658ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9") failed: Property not found. retry in 675.881281ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9
11192024-08-30T19:42:28.670ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_431dc29b-e656-4e8d-af66-a8f30ed60f1a") failed: Property not found. retry in 1.014169287s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_431dc29b-e656-4e8d-af66-a8f30ed60f1a
11202024-08-30T19:42:28.680ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_ff2478ea-0538-402c-8476-a8f41b89a1dd") failed: Property not found. retry in 1.232410148s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_ff2478ea-0538-402c-8476-a8f41b89a1dd
11212024-08-30T19:42:28.781ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_42b63708-3cd8-4e58-9c6d-786078ef5ea3") failed: Property not found. retry in 1.314325642s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_42b63708-3cd8-4e58-9c6d-786078ef5ea3
11222024-08-30T19:42:28.941ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ba4aef74-7b33-4ded-a46a-dc2534b0db9d") failed: Property not found. retry in 610.389159ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ba4aef74-7b33-4ded-a46a-dc2534b0db9d
11232024-08-30T19:42:28.957ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c79a17e-c108-4cbb-9ac6-cc88acef9672") failed: Property not found. retry in 973.739424ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c79a17e-c108-4cbb-9ac6-cc88acef9672
11242024-08-30T19:42:28.977ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e1f7623b-2d1f-4b94-b59f-b88843322bd4") failed: Property not found. retry in 1.005901892s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e1f7623b-2d1f-4b94-b59f-b88843322bd4
11252024-08-30T19:42:29.065ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d2343105-862a-478c-b746-fd8bb78f9a96") failed: Property not found. retry in 1.129789993s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d2343105-862a-478c-b746-fd8bb78f9a96
11262024-08-30T19:42:29.078ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1d96d476-acdb-4c44-a486-29569b3bf41e") failed: Property not found. retry in 1.404921652s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1d96d476-acdb-4c44-a486-29569b3bf41e
11272024-08-30T19:42:29.134ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c65dc850-b02e-4247-904b-870d82350626") failed: Property not found. retry in 1.382635363s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c65dc850-b02e-4247-904b-870d82350626
11282024-08-30T19:42:29.160ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_b5636b10-76ec-4aa4-8bd4-1f95780813df") failed: Property not found. retry in 846.93435ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_b5636b10-76ec-4aa4-8bd4-1f95780813df
11292024-08-30T19:42:29.269ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_391744bd-9064-4e92-ab26-8edda407dd8c") failed: Property not found. retry in 1.470315094s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_391744bd-9064-4e92-ab26-8edda407dd8c
11302024-08-30T19:42:29.313ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6651a7a2-6c34-40eb-8170-3be50047e5a1") failed: Property not found. retry in 1.093211578s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6651a7a2-6c34-40eb-8170-3be50047e5a1
11312024-08-30T19:42:29.348ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9") failed: Property not found. retry in 722.520452ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9
11322024-08-30T19:42:29.423ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ad118dce-b2fc-49ed-9fb7-9a53178d3588") failed: Property not found. retry in 692.220585ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ad118dce-b2fc-49ed-9fb7-9a53178d3588
11332024-08-30T19:42:29.444ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_a472c33d-180a-48f1-9999-d170d3a97f4c") failed: Property not found. retry in 791.554297ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_a472c33d-180a-48f1-9999-d170d3a97f4c
11342024-08-30T19:42:29.651ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b7932671-3c29-4be4-828c-4ccf13631712") failed: Property not found. retry in 1.135426951s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b7932671-3c29-4be4-828c-4ccf13631712
11352024-08-30T19:42:29.700ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ba4aef74-7b33-4ded-a46a-dc2534b0db9d") failed: Property not found. retry in 518.128596ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ba4aef74-7b33-4ded-a46a-dc2534b0db9d
11362024-08-30T19:42:29.726ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_431dc29b-e656-4e8d-af66-a8f30ed60f1a") failed: Property not found. retry in 579.29112ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_431dc29b-e656-4e8d-af66-a8f30ed60f1a
11372024-08-30T19:42:29.790ZINFOSledAgent (ServiceManager): Profile for oxz_external_dns_ff52340f-525a-4b88-bfc8-1cb0fe9ceba7: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService28'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::8'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte5'/> <propval type="astring" name="gateway" value='172.30.1.1'/> <propval type="astring" name="ip" value='172.30.1.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/external_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::8]:5353'/> <propval type="astring" name="dns_address" value='172.30.1.5:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11382024-08-30T19:42:29.790ZINFOSledAgent (ServiceManager): Booting oxz_external_dns_ff52340f-525a-4b88-bfc8-1cb0fe9ceba7 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_external_dns_ff52340f-525a-4b88-bfc8-1cb0fe9ceba7
11392024-08-30T19:42:29.928ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_ff2478ea-0538-402c-8476-a8f41b89a1dd") failed: Property not found. retry in 643.704565ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_ff2478ea-0538-402c-8476-a8f41b89a1dd
11402024-08-30T19:42:29.946ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66") failed: Property not found. retry in 727.58016ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66
11412024-08-30T19:42:29.958ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c79a17e-c108-4cbb-9ac6-cc88acef9672") failed: Property not found. retry in 1.450953432s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c79a17e-c108-4cbb-9ac6-cc88acef9672
11422024-08-30T19:42:29.998ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e1f7623b-2d1f-4b94-b59f-b88843322bd4") failed: Property not found. retry in 1.36151273s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e1f7623b-2d1f-4b94-b59f-b88843322bd4
11432024-08-30T19:42:30.032ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_b5636b10-76ec-4aa4-8bd4-1f95780813df") failed: Property not found. retry in 1.351871959s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_b5636b10-76ec-4aa4-8bd4-1f95780813df
11442024-08-30T19:42:30.088ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9") failed: Property not found. retry in 1.043362639s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9
11452024-08-30T19:42:30.099ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_ff52340f-525a-4b88-bfc8-1cb0fe9ceba7") failed: Property not found. retry in 52.808671ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_ff52340f-525a-4b88-bfc8-1cb0fe9ceba7
11462024-08-30T19:42:30.112ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_42b63708-3cd8-4e58-9c6d-786078ef5ea3") failed: Property not found. retry in 724.951028ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_42b63708-3cd8-4e58-9c6d-786078ef5ea3
11472024-08-30T19:42:30.130ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ad118dce-b2fc-49ed-9fb7-9a53178d3588") failed: Property not found. retry in 778.114512ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ad118dce-b2fc-49ed-9fb7-9a53178d3588
11482024-08-30T19:42:30.164ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_ff52340f-525a-4b88-bfc8-1cb0fe9ceba7") failed: Property not found. retry in 97.494828ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_ff52340f-525a-4b88-bfc8-1cb0fe9ceba7
11492024-08-30T19:42:30.209ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d2343105-862a-478c-b746-fd8bb78f9a96") failed: Property not found. retry in 567.263465ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d2343105-862a-478c-b746-fd8bb78f9a96
11502024-08-30T19:42:30.233ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ba4aef74-7b33-4ded-a46a-dc2534b0db9d") failed: Property not found. retry in 1.451715617s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ba4aef74-7b33-4ded-a46a-dc2534b0db9d
11512024-08-30T19:42:30.250ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_a472c33d-180a-48f1-9999-d170d3a97f4c") failed: Property not found. retry in 678.566782ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_a472c33d-180a-48f1-9999-d170d3a97f4c
11522024-08-30T19:42:30.275ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_ff52340f-525a-4b88-bfc8-1cb0fe9ceba7") failed: Property not found. retry in 221.387152ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_ff52340f-525a-4b88-bfc8-1cb0fe9ceba7
11532024-08-30T19:42:30.321ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_431dc29b-e656-4e8d-af66-a8f30ed60f1a") failed: Property not found. retry in 1.475763303s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_431dc29b-e656-4e8d-af66-a8f30ed60f1a
11542024-08-30T19:42:30.424ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6651a7a2-6c34-40eb-8170-3be50047e5a1") failed: Property not found. retry in 730.819995ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6651a7a2-6c34-40eb-8170-3be50047e5a1
11552024-08-30T19:42:30.502ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1d96d476-acdb-4c44-a486-29569b3bf41e") failed: Property not found. retry in 1.324347952s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1d96d476-acdb-4c44-a486-29569b3bf41e
11562024-08-30T19:42:30.523ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_ff52340f-525a-4b88-bfc8-1cb0fe9ceba7") failed: Property not found. retry in 388.09549ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_ff52340f-525a-4b88-bfc8-1cb0fe9ceba7
11572024-08-30T19:42:30.550ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c65dc850-b02e-4247-904b-870d82350626") failed: Property not found. retry in 1.119969383s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c65dc850-b02e-4247-904b-870d82350626
11582024-08-30T19:42:30.672ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_ff2478ea-0538-402c-8476-a8f41b89a1dd") failed: Property not found. retry in 957.790641ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_ff2478ea-0538-402c-8476-a8f41b89a1dd
11592024-08-30T19:42:30.703ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66") failed: Property not found. retry in 888.383407ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66
11602024-08-30T19:42:30.786ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_391744bd-9064-4e92-ab26-8edda407dd8c") failed: Property not found. retry in 957.623748ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_391744bd-9064-4e92-ab26-8edda407dd8c
11612024-08-30T19:42:30.814ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d2343105-862a-478c-b746-fd8bb78f9a96") failed: Property not found. retry in 1.152076372s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d2343105-862a-478c-b746-fd8bb78f9a96
11622024-08-30T19:42:30.841ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b7932671-3c29-4be4-828c-4ccf13631712") failed: Property not found. retry in 865.183669ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b7932671-3c29-4be4-828c-4ccf13631712
11632024-08-30T19:42:30.878ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_42b63708-3cd8-4e58-9c6d-786078ef5ea3") failed: Property not found. retry in 1.023661711s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_42b63708-3cd8-4e58-9c6d-786078ef5ea3
11642024-08-30T19:42:30.928ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ad118dce-b2fc-49ed-9fb7-9a53178d3588") failed: Property not found. retry in 1.164215102s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ad118dce-b2fc-49ed-9fb7-9a53178d3588
11652024-08-30T19:42:30.951ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_ff52340f-525a-4b88-bfc8-1cb0fe9ceba7") failed: Property not found. retry in 1.125998416s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_ff52340f-525a-4b88-bfc8-1cb0fe9ceba7
11662024-08-30T19:42:30.991ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_a472c33d-180a-48f1-9999-d170d3a97f4c") failed: Property not found. retry in 643.767571ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_a472c33d-180a-48f1-9999-d170d3a97f4c
11672024-08-30T19:42:31.145ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9") failed: Property not found. retry in 978.018491ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9
11682024-08-30T19:42:31.169ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6651a7a2-6c34-40eb-8170-3be50047e5a1") failed: Property not found. retry in 1.180151546s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6651a7a2-6c34-40eb-8170-3be50047e5a1
11692024-08-30T19:42:31.372ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e1f7623b-2d1f-4b94-b59f-b88843322bd4") failed: Property not found. retry in 1.090024094s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e1f7623b-2d1f-4b94-b59f-b88843322bd4
11702024-08-30T19:42:31.400ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_b5636b10-76ec-4aa4-8bd4-1f95780813df") failed: Property not found. retry in 1.109585083s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_b5636b10-76ec-4aa4-8bd4-1f95780813df
11712024-08-30T19:42:31.423ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c79a17e-c108-4cbb-9ac6-cc88acef9672") failed: Property not found. retry in 920.523128ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c79a17e-c108-4cbb-9ac6-cc88acef9672
11722024-08-30T19:42:31.615ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66") failed: Property not found. retry in 684.674031ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66
11732024-08-30T19:42:31.653ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_ff2478ea-0538-402c-8476-a8f41b89a1dd") failed: Property not found. retry in 989.203617ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_ff2478ea-0538-402c-8476-a8f41b89a1dd
11742024-08-30T19:42:31.676ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_a472c33d-180a-48f1-9999-d170d3a97f4c") failed: Property not found. retry in 734.785798ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_a472c33d-180a-48f1-9999-d170d3a97f4c
11752024-08-30T19:42:31.698ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c65dc850-b02e-4247-904b-870d82350626") failed: Property not found. retry in 1.214258478s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c65dc850-b02e-4247-904b-870d82350626
11762024-08-30T19:42:31.735ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ba4aef74-7b33-4ded-a46a-dc2534b0db9d") failed: Property not found. retry in 1.423393543s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ba4aef74-7b33-4ded-a46a-dc2534b0db9d
11772024-08-30T19:42:31.772ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b7932671-3c29-4be4-828c-4ccf13631712") failed: Property not found. retry in 668.620053ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b7932671-3c29-4be4-828c-4ccf13631712
11782024-08-30T19:42:31.793ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_391744bd-9064-4e92-ab26-8edda407dd8c") failed: Property not found. retry in 684.938536ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_391744bd-9064-4e92-ab26-8edda407dd8c
11792024-08-30T19:42:31.815ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_431dc29b-e656-4e8d-af66-a8f30ed60f1a") failed: Property not found. retry in 1.386884867s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_431dc29b-e656-4e8d-af66-a8f30ed60f1a
11802024-08-30T19:42:31.850ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1d96d476-acdb-4c44-a486-29569b3bf41e") failed: Property not found. retry in 518.267967ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1d96d476-acdb-4c44-a486-29569b3bf41e
11812024-08-30T19:42:31.918ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_42b63708-3cd8-4e58-9c6d-786078ef5ea3") failed: Property not found. retry in 1.00201593s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_42b63708-3cd8-4e58-9c6d-786078ef5ea3
11822024-08-30T19:42:31.984ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d2343105-862a-478c-b746-fd8bb78f9a96") failed: Property not found. retry in 1.422264939s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d2343105-862a-478c-b746-fd8bb78f9a96
11832024-08-30T19:42:32.094ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_ff52340f-525a-4b88-bfc8-1cb0fe9ceba7") failed: Property not found. retry in 883.613826ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_ff52340f-525a-4b88-bfc8-1cb0fe9ceba7
11842024-08-30T19:42:32.112ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ad118dce-b2fc-49ed-9fb7-9a53178d3588") failed: Property not found. retry in 779.010194ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ad118dce-b2fc-49ed-9fb7-9a53178d3588
11852024-08-30T19:42:32.139ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9") failed: Property not found. retry in 837.455325ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9
11862024-08-30T19:42:32.330ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66") failed: Property not found. retry in 1.431644389s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66
11872024-08-30T19:42:32.365ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c79a17e-c108-4cbb-9ac6-cc88acef9672") failed: Property not found. retry in 1.009311682s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c79a17e-c108-4cbb-9ac6-cc88acef9672
11882024-08-30T19:42:32.386ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6651a7a2-6c34-40eb-8170-3be50047e5a1") failed: Property not found. retry in 1.033023854s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6651a7a2-6c34-40eb-8170-3be50047e5a1
11892024-08-30T19:42:32.404ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1d96d476-acdb-4c44-a486-29569b3bf41e") failed: Property not found. retry in 1.008707927s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1d96d476-acdb-4c44-a486-29569b3bf41e
11902024-08-30T19:42:32.429ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_a472c33d-180a-48f1-9999-d170d3a97f4c") failed: Property not found. retry in 1.339897007s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_a472c33d-180a-48f1-9999-d170d3a97f4c
11912024-08-30T19:42:32.462ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b7932671-3c29-4be4-828c-4ccf13631712") failed: Property not found. retry in 685.192979ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b7932671-3c29-4be4-828c-4ccf13631712
11922024-08-30T19:42:32.487ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e1f7623b-2d1f-4b94-b59f-b88843322bd4") failed: Property not found. retry in 646.900881ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e1f7623b-2d1f-4b94-b59f-b88843322bd4
11932024-08-30T19:42:32.506ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_391744bd-9064-4e92-ab26-8edda407dd8c") failed: Property not found. retry in 919.079359ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_391744bd-9064-4e92-ab26-8edda407dd8c
11942024-08-30T19:42:32.544ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_b5636b10-76ec-4aa4-8bd4-1f95780813df") failed: Property not found. retry in 1.261084051s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_b5636b10-76ec-4aa4-8bd4-1f95780813df
11952024-08-30T19:42:32.663ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_ff2478ea-0538-402c-8476-a8f41b89a1dd") failed: Property not found. retry in 1.242495919s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_ff2478ea-0538-402c-8476-a8f41b89a1dd
11962024-08-30T19:42:32.903ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ad118dce-b2fc-49ed-9fb7-9a53178d3588") failed: Property not found. retry in 909.90327ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ad118dce-b2fc-49ed-9fb7-9a53178d3588
11972024-08-30T19:42:32.925ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c65dc850-b02e-4247-904b-870d82350626") failed: Property not found. retry in 1.1125474s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c65dc850-b02e-4247-904b-870d82350626
11982024-08-30T19:42:32.938ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_42b63708-3cd8-4e58-9c6d-786078ef5ea3") failed: Property not found. retry in 565.511929ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_42b63708-3cd8-4e58-9c6d-786078ef5ea3
11992024-08-30T19:42:32.988ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9") failed: Property not found. retry in 647.512777ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9
12002024-08-30T19:42:32.999ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_ff52340f-525a-4b88-bfc8-1cb0fe9ceba7") failed: Property not found. retry in 726.548053ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_ff52340f-525a-4b88-bfc8-1cb0fe9ceba7
12012024-08-30T19:42:33.149ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e1f7623b-2d1f-4b94-b59f-b88843322bd4") failed: Property not found. retry in 1.038431106s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e1f7623b-2d1f-4b94-b59f-b88843322bd4
12022024-08-30T19:42:33.160ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b7932671-3c29-4be4-828c-4ccf13631712") failed: Property not found. retry in 1.375318193s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b7932671-3c29-4be4-828c-4ccf13631712
12032024-08-30T19:42:33.170ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ba4aef74-7b33-4ded-a46a-dc2534b0db9d") failed: Property not found. retry in 1.109975934s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ba4aef74-7b33-4ded-a46a-dc2534b0db9d
12042024-08-30T19:42:33.214ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_431dc29b-e656-4e8d-af66-a8f30ed60f1a") failed: Property not found. retry in 1.499221231s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_431dc29b-e656-4e8d-af66-a8f30ed60f1a
12052024-08-30T19:42:33.388ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c79a17e-c108-4cbb-9ac6-cc88acef9672") failed: Property not found. retry in 1.056681373s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c79a17e-c108-4cbb-9ac6-cc88acef9672
12062024-08-30T19:42:33.418ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d2343105-862a-478c-b746-fd8bb78f9a96") failed: Property not found. retry in 1.139175635s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d2343105-862a-478c-b746-fd8bb78f9a96
12072024-08-30T19:42:33.428ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1d96d476-acdb-4c44-a486-29569b3bf41e") failed: Property not found. retry in 862.425669ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1d96d476-acdb-4c44-a486-29569b3bf41e
12082024-08-30T19:42:33.438ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6651a7a2-6c34-40eb-8170-3be50047e5a1") failed: Property not found. retry in 843.991313ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6651a7a2-6c34-40eb-8170-3be50047e5a1
12092024-08-30T19:42:33.450ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_391744bd-9064-4e92-ab26-8edda407dd8c") failed: Property not found. retry in 662.795173ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_391744bd-9064-4e92-ab26-8edda407dd8c
12102024-08-30T19:42:33.514ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_42b63708-3cd8-4e58-9c6d-786078ef5ea3") failed: Property not found. retry in 1.453199046s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_42b63708-3cd8-4e58-9c6d-786078ef5ea3
12112024-08-30T19:42:33.647ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9") failed: Property not found. retry in 703.490223ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9
12122024-08-30T19:42:33.737ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_ff52340f-525a-4b88-bfc8-1cb0fe9ceba7") failed: Property not found. retry in 1.224298102s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_ff52340f-525a-4b88-bfc8-1cb0fe9ceba7
12132024-08-30T19:42:33.773ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 2.912877086s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be
12142024-08-30T19:42:33.785ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66") failed: Property not found. retry in 979.543776ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66
12152024-08-30T19:42:33.796ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_a472c33d-180a-48f1-9999-d170d3a97f4c") failed: Property not found. retry in 773.26826ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_a472c33d-180a-48f1-9999-d170d3a97f4c
12162024-08-30T19:42:33.818ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_b5636b10-76ec-4aa4-8bd4-1f95780813df") failed: Property not found. retry in 912.120689ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_b5636b10-76ec-4aa4-8bd4-1f95780813df
12172024-08-30T19:42:33.829ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ad118dce-b2fc-49ed-9fb7-9a53178d3588") failed: Property not found. retry in 536.87944ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ad118dce-b2fc-49ed-9fb7-9a53178d3588
12182024-08-30T19:42:33.915ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_ff2478ea-0538-402c-8476-a8f41b89a1dd") failed: Property not found. retry in 513.990474ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_ff2478ea-0538-402c-8476-a8f41b89a1dd
12192024-08-30T19:42:34.051ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c65dc850-b02e-4247-904b-870d82350626") failed: Property not found. retry in 569.103908ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c65dc850-b02e-4247-904b-870d82350626
12202024-08-30T19:42:34.100ZINFOSledAgent (ServiceManager): Profile for oxz_clickhouse_7abe137b-8d6c-4edb-b5cb-88d7f97fb593: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService13'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::e'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/clickhouse"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::e'/> <propval type="astring" name="listen_port" value='8123'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/clickhouse-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="clickhouse_address" value='[fd00:1122:3344:101::e]:8123'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::e]:8888'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
12212024-08-30T19:42:34.101ZINFOSledAgent (ServiceManager): Booting oxz_clickhouse_7abe137b-8d6c-4edb-b5cb-88d7f97fb593 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_clickhouse_7abe137b-8d6c-4edb-b5cb-88d7f97fb593
12222024-08-30T19:42:34.124ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_391744bd-9064-4e92-ab26-8edda407dd8c") failed: Property not found. retry in 833.383628ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_391744bd-9064-4e92-ab26-8edda407dd8c
12232024-08-30T19:42:34.200ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e1f7623b-2d1f-4b94-b59f-b88843322bd4") failed: Property not found. retry in 1.133663723s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e1f7623b-2d1f-4b94-b59f-b88843322bd4
12242024-08-30T19:42:34.294ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ba4aef74-7b33-4ded-a46a-dc2534b0db9d") failed: Property not found. retry in 871.733446ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ba4aef74-7b33-4ded-a46a-dc2534b0db9d
12252024-08-30T19:42:34.304ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1d96d476-acdb-4c44-a486-29569b3bf41e") failed: Property not found. retry in 1.193698214s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1d96d476-acdb-4c44-a486-29569b3bf41e
12262024-08-30T19:42:34.315ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6651a7a2-6c34-40eb-8170-3be50047e5a1") failed: Property not found. retry in 1.489270097s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6651a7a2-6c34-40eb-8170-3be50047e5a1
12272024-08-30T19:42:34.340ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_7abe137b-8d6c-4edb-b5cb-88d7f97fb593") failed: Property not found. retry in 61.928574ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_7abe137b-8d6c-4edb-b5cb-88d7f97fb593
12282024-08-30T19:42:34.365ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9") failed: Property not found. retry in 644.852903ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9
12292024-08-30T19:42:34.377ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ad118dce-b2fc-49ed-9fb7-9a53178d3588") failed: Property not found. retry in 760.513812ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ad118dce-b2fc-49ed-9fb7-9a53178d3588
12302024-08-30T19:42:34.414ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_7abe137b-8d6c-4edb-b5cb-88d7f97fb593") failed: Property not found. retry in 149.897279ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_7abe137b-8d6c-4edb-b5cb-88d7f97fb593
12312024-08-30T19:42:34.624ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_ff2478ea-0538-402c-8476-a8f41b89a1dd") failed: Property not found. retry in 925.262215ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_ff2478ea-0538-402c-8476-a8f41b89a1dd
12322024-08-30T19:42:34.718ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b7932671-3c29-4be4-828c-4ccf13631712") failed: Property not found. retry in 1.350471079s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b7932671-3c29-4be4-828c-4ccf13631712
12332024-08-30T19:42:34.734ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_431dc29b-e656-4e8d-af66-a8f30ed60f1a") failed: Property not found. retry in 975.197256ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_431dc29b-e656-4e8d-af66-a8f30ed60f1a
12342024-08-30T19:42:34.753ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_7abe137b-8d6c-4edb-b5cb-88d7f97fb593") failed: Property not found. retry in 273.331765ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_7abe137b-8d6c-4edb-b5cb-88d7f97fb593
12352024-08-30T19:42:34.770ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_b5636b10-76ec-4aa4-8bd4-1f95780813df") failed: Property not found. retry in 510.381071ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_b5636b10-76ec-4aa4-8bd4-1f95780813df
12362024-08-30T19:42:34.783ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66") failed: Property not found. retry in 765.049615ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66
12372024-08-30T19:42:34.797ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d2343105-862a-478c-b746-fd8bb78f9a96") failed: Property not found. retry in 1.027802286s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d2343105-862a-478c-b746-fd8bb78f9a96
12382024-08-30T19:42:34.811ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c65dc850-b02e-4247-904b-870d82350626") failed: Property not found. retry in 1.498007721s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c65dc850-b02e-4247-904b-870d82350626
12392024-08-30T19:42:34.823ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_a472c33d-180a-48f1-9999-d170d3a97f4c") failed: Property not found. retry in 952.86316ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_a472c33d-180a-48f1-9999-d170d3a97f4c
12402024-08-30T19:42:34.837ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c79a17e-c108-4cbb-9ac6-cc88acef9672") failed: Property not found. retry in 592.716759ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c79a17e-c108-4cbb-9ac6-cc88acef9672
12412024-08-30T19:42:34.969ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_391744bd-9064-4e92-ab26-8edda407dd8c") failed: Property not found. retry in 835.753487ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_391744bd-9064-4e92-ab26-8edda407dd8c
12422024-08-30T19:42:34.982ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_42b63708-3cd8-4e58-9c6d-786078ef5ea3") failed: Property not found. retry in 1.314266464s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_42b63708-3cd8-4e58-9c6d-786078ef5ea3
12432024-08-30T19:42:34.993ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_ff52340f-525a-4b88-bfc8-1cb0fe9ceba7") failed: Property not found. retry in 1.240492067s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_ff52340f-525a-4b88-bfc8-1cb0fe9ceba7
12442024-08-30T19:42:35.021ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9") failed: Property not found. retry in 790.589098ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9
12452024-08-30T19:42:35.040ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_7abe137b-8d6c-4edb-b5cb-88d7f97fb593") failed: Property not found. retry in 393.574477ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_7abe137b-8d6c-4edb-b5cb-88d7f97fb593
12462024-08-30T19:42:35.153ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ad118dce-b2fc-49ed-9fb7-9a53178d3588") failed: Property not found. retry in 1.097361251s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ad118dce-b2fc-49ed-9fb7-9a53178d3588
12472024-08-30T19:42:35.179ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ba4aef74-7b33-4ded-a46a-dc2534b0db9d") failed: Property not found. retry in 1.08424838s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ba4aef74-7b33-4ded-a46a-dc2534b0db9d
12482024-08-30T19:42:35.296ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_b5636b10-76ec-4aa4-8bd4-1f95780813df") failed: Property not found. retry in 626.054979ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_b5636b10-76ec-4aa4-8bd4-1f95780813df
12492024-08-30T19:42:35.347ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e1f7623b-2d1f-4b94-b59f-b88843322bd4") failed: Property not found. retry in 1.375668862s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e1f7623b-2d1f-4b94-b59f-b88843322bd4
12502024-08-30T19:42:35.444ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c79a17e-c108-4cbb-9ac6-cc88acef9672") failed: Property not found. retry in 779.604377ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c79a17e-c108-4cbb-9ac6-cc88acef9672
12512024-08-30T19:42:35.461ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_7abe137b-8d6c-4edb-b5cb-88d7f97fb593") failed: Property not found. retry in 909.448248ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_7abe137b-8d6c-4edb-b5cb-88d7f97fb593
12522024-08-30T19:42:35.515ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1d96d476-acdb-4c44-a486-29569b3bf41e") failed: Property not found. retry in 802.509511ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1d96d476-acdb-4c44-a486-29569b3bf41e
12532024-08-30T19:42:35.564ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66") failed: Property not found. retry in 796.273314ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66
12542024-08-30T19:42:35.576ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_ff2478ea-0538-402c-8476-a8f41b89a1dd") failed: Property not found. retry in 836.716291ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_ff2478ea-0538-402c-8476-a8f41b89a1dd
12552024-08-30T19:42:35.725ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_431dc29b-e656-4e8d-af66-a8f30ed60f1a") failed: Property not found. retry in 737.081912ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_431dc29b-e656-4e8d-af66-a8f30ed60f1a
12562024-08-30T19:42:35.792ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_a472c33d-180a-48f1-9999-d170d3a97f4c") failed: Property not found. retry in 1.153816561s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_a472c33d-180a-48f1-9999-d170d3a97f4c
12572024-08-30T19:42:35.816ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6651a7a2-6c34-40eb-8170-3be50047e5a1") failed: Property not found. retry in 1.300569899s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6651a7a2-6c34-40eb-8170-3be50047e5a1
12582024-08-30T19:42:35.829ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9") failed: Property not found. retry in 1.46964166s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9
12592024-08-30T19:42:35.845ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_391744bd-9064-4e92-ab26-8edda407dd8c") failed: Property not found. retry in 534.409378ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_391744bd-9064-4e92-ab26-8edda407dd8c
12602024-08-30T19:42:35.863ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d2343105-862a-478c-b746-fd8bb78f9a96") failed: Property not found. retry in 1.403029795s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d2343105-862a-478c-b746-fd8bb78f9a96
12612024-08-30T19:42:35.938ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_b5636b10-76ec-4aa4-8bd4-1f95780813df") failed: Property not found. retry in 653.158355ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_b5636b10-76ec-4aa4-8bd4-1f95780813df
12622024-08-30T19:42:36.085ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b7932671-3c29-4be4-828c-4ccf13631712") failed: Property not found. retry in 1.278940371s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b7932671-3c29-4be4-828c-4ccf13631712
12632024-08-30T19:42:36.239ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c79a17e-c108-4cbb-9ac6-cc88acef9672") failed: Property not found. retry in 1.058878294s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c79a17e-c108-4cbb-9ac6-cc88acef9672
12642024-08-30T19:42:36.254ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_ff52340f-525a-4b88-bfc8-1cb0fe9ceba7") failed: Property not found. retry in 1.04946155s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_ff52340f-525a-4b88-bfc8-1cb0fe9ceba7
12652024-08-30T19:42:36.271ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ad118dce-b2fc-49ed-9fb7-9a53178d3588") failed: Property not found. retry in 875.959567ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ad118dce-b2fc-49ed-9fb7-9a53178d3588
12662024-08-30T19:42:36.285ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ba4aef74-7b33-4ded-a46a-dc2534b0db9d") failed: Property not found. retry in 1.338069009s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ba4aef74-7b33-4ded-a46a-dc2534b0db9d
12672024-08-30T19:42:36.311ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_42b63708-3cd8-4e58-9c6d-786078ef5ea3") failed: Property not found. retry in 973.995765ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_42b63708-3cd8-4e58-9c6d-786078ef5ea3
12682024-08-30T19:42:36.327ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c65dc850-b02e-4247-904b-870d82350626") failed: Property not found. retry in 1.000878351s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c65dc850-b02e-4247-904b-870d82350626
12692024-08-30T19:42:36.341ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1d96d476-acdb-4c44-a486-29569b3bf41e") failed: Property not found. retry in 929.560752ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1d96d476-acdb-4c44-a486-29569b3bf41e
12702024-08-30T19:42:36.376ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66") failed: Property not found. retry in 570.901981ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66
12712024-08-30T19:42:36.391ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_7abe137b-8d6c-4edb-b5cb-88d7f97fb593") failed: Property not found. retry in 1.467177712s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_7abe137b-8d6c-4edb-b5cb-88d7f97fb593
12722024-08-30T19:42:36.405ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_391744bd-9064-4e92-ab26-8edda407dd8c") failed: Property not found. retry in 1.134712887s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_391744bd-9064-4e92-ab26-8edda407dd8c
12732024-08-30T19:42:36.431ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_ff2478ea-0538-402c-8476-a8f41b89a1dd") failed: Property not found. retry in 1.015746383s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_ff2478ea-0538-402c-8476-a8f41b89a1dd
12742024-08-30T19:42:36.479ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_431dc29b-e656-4e8d-af66-a8f30ed60f1a") failed: Property not found. retry in 709.203113ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_431dc29b-e656-4e8d-af66-a8f30ed60f1a
12752024-08-30T19:42:36.611ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_b5636b10-76ec-4aa4-8bd4-1f95780813df") failed: Property not found. retry in 631.562821ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_b5636b10-76ec-4aa4-8bd4-1f95780813df
12762024-08-30T19:42:36.969ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_a472c33d-180a-48f1-9999-d170d3a97f4c") failed: Property not found. retry in 1.460396086s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_a472c33d-180a-48f1-9999-d170d3a97f4c
12772024-08-30T19:42:36.995ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66") failed: Property not found. retry in 1.491478479s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66
12782024-08-30T19:42:37.265ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_b5636b10-76ec-4aa4-8bd4-1f95780813df") failed: Property not found. retry in 556.227906ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_b5636b10-76ec-4aa4-8bd4-1f95780813df
12792024-08-30T19:42:37.291ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d2343105-862a-478c-b746-fd8bb78f9a96") failed: Property not found. retry in 1.429315852s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d2343105-862a-478c-b746-fd8bb78f9a96
12802024-08-30T19:42:37.313ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_42b63708-3cd8-4e58-9c6d-786078ef5ea3") failed: Property not found. retry in 916.122949ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_42b63708-3cd8-4e58-9c6d-786078ef5ea3
12812024-08-30T19:42:37.350ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1d96d476-acdb-4c44-a486-29569b3bf41e") failed: Property not found. retry in 1.348649703s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1d96d476-acdb-4c44-a486-29569b3bf41e
12822024-08-30T19:42:37.426ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_ff52340f-525a-4b88-bfc8-1cb0fe9ceba7") failed: Property not found. retry in 709.256854ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_ff52340f-525a-4b88-bfc8-1cb0fe9ceba7
12832024-08-30T19:42:37.649ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9") failed: Property not found. retry in 1.113886667s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9
12842024-08-30T19:42:37.790ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_ff2478ea-0538-402c-8476-a8f41b89a1dd") failed: Property not found. retry in 1.149214325s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_ff2478ea-0538-402c-8476-a8f41b89a1dd
12852024-08-30T19:42:37.884ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_7abe137b-8d6c-4edb-b5cb-88d7f97fb593") failed: Property not found. retry in 993.888627ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_7abe137b-8d6c-4edb-b5cb-88d7f97fb593
12862024-08-30T19:42:38.163ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_ff52340f-525a-4b88-bfc8-1cb0fe9ceba7") failed: Property not found. retry in 830.986813ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_ff52340f-525a-4b88-bfc8-1cb0fe9ceba7
12872024-08-30T19:42:38.208ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 2.054418472s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers): error trying to connect: tcp connect error: Connection refused (os error 146)" file = oximeter/producer/src/lib.rs:424 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be
12882024-08-30T19:42:38.457ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_a472c33d-180a-48f1-9999-d170d3a97f4c") failed: Property not found. retry in 1.402891848s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_a472c33d-180a-48f1-9999-d170d3a97f4c
12892024-08-30T19:42:38.515ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66") failed: Property not found. retry in 1.030612182s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66
12902024-08-30T19:42:38.788ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9") failed: Property not found. retry in 1.49679711s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9
12912024-08-30T19:42:38.901ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_7abe137b-8d6c-4edb-b5cb-88d7f97fb593") failed: Property not found. retry in 665.998922ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_7abe137b-8d6c-4edb-b5cb-88d7f97fb593
12922024-08-30T19:42:39.014ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_ff52340f-525a-4b88-bfc8-1cb0fe9ceba7") failed: Property not found. retry in 1.277394741s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_ff52340f-525a-4b88-bfc8-1cb0fe9ceba7
12932024-08-30T19:42:39.564ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66") failed: Property not found. retry in 795.979038ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66
12942024-08-30T19:42:39.581ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_7abe137b-8d6c-4edb-b5cb-88d7f97fb593") failed: Property not found. retry in 977.893192ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_7abe137b-8d6c-4edb-b5cb-88d7f97fb593
12952024-08-30T19:42:40.333ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 4.120285348s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers): error trying to connect: tcp connect error: Connection refused (os error 146)" file = oximeter/producer/src/lib.rs:424 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be
12962024-08-30T19:42:40.360ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_ff52340f-525a-4b88-bfc8-1cb0fe9ceba7") failed: Property not found. retry in 932.027463ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_ff52340f-525a-4b88-bfc8-1cb0fe9ceba7
12972024-08-30T19:42:40.372ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66") failed: Property not found. retry in 843.032331ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66
12982024-08-30T19:42:40.576ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_7abe137b-8d6c-4edb-b5cb-88d7f97fb593") failed: Property not found. retry in 1.169917715s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_7abe137b-8d6c-4edb-b5cb-88d7f97fb593
12992024-08-30T19:42:41.224ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66") failed: Property not found. retry in 680.169863ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66
13002024-08-30T19:42:41.303ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_ff52340f-525a-4b88-bfc8-1cb0fe9ceba7") failed: Property not found. retry in 1.185463246s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_ff52340f-525a-4b88-bfc8-1cb0fe9ceba7
13012024-08-30T19:42:41.609ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:46849
13022024-08-30T19:42:41.633ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 23955 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:46849 req_id = 2991086b-dbce-44ae-b1ea-c3d995992e69 response_code = 200 uri = /baseboard
13032024-08-30T19:42:41.756ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_7abe137b-8d6c-4edb-b5cb-88d7f97fb593") failed: Property not found. retry in 738.767008ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_7abe137b-8d6c-4edb-b5cb-88d7f97fb593
13042024-08-30T19:42:42.514ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_7abe137b-8d6c-4edb-b5cb-88d7f97fb593") failed: Property not found. retry in 1.044533474s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_7abe137b-8d6c-4edb-b5cb-88d7f97fb593
13052024-08-30T19:42:43.569ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_7abe137b-8d6c-4edb-b5cb-88d7f97fb593") failed: Property not found. retry in 915.241076ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_7abe137b-8d6c-4edb-b5cb-88d7f97fb593
13062024-08-30T19:42:44.498ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 12.448951589s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers): error trying to connect: tcp connect error: Connection refused (os error 146)" file = oximeter/producer/src/lib.rs:424 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be
13072024-08-30T19:42:44.516ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_crucible_b7932671-3c29-4be4-828c-4ccf13631712
13082024-08-30T19:42:44.516ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_crucible_431dc29b-e656-4e8d-af66-a8f30ed60f1a
13092024-08-30T19:42:44.517ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_crucible_f0ee96d9-e31a-49fe-8fdd-b5786e918bd9
13102024-08-30T19:42:44.528ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_clickhouse_7abe137b-8d6c-4edb-b5cb-88d7f97fb593
13112024-08-30T19:42:44.528ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_crucible_42b63708-3cd8-4e58-9c6d-786078ef5ea3
13122024-08-30T19:42:44.528ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_nexus_1d96d476-acdb-4c44-a486-29569b3bf41e
13132024-08-30T19:42:44.528ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_external_dns_b5636b10-76ec-4aa4-8bd4-1f95780813df
13142024-08-30T19:42:44.528ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_crucible_pantry_6651a7a2-6c34-40eb-8170-3be50047e5a1
13152024-08-30T19:42:44.528ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_crucible_ba6c932e-9a46-4c12-8544-9c868066cb66
13162024-08-30T19:42:44.528ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_crucible_ad118dce-b2fc-49ed-9fb7-9a53178d3588
13172024-08-30T19:42:44.528ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_oximeter_391744bd-9064-4e92-ab26-8edda407dd8c
13182024-08-30T19:42:44.528ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_nexus_d2343105-862a-478c-b746-fd8bb78f9a96
13192024-08-30T19:42:44.528ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_crucible_c65dc850-b02e-4247-904b-870d82350626
13202024-08-30T19:42:44.528ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_crucible_pantry_a472c33d-180a-48f1-9999-d170d3a97f4c
13212024-08-30T19:42:44.528ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_nexus_ba4aef74-7b33-4ded-a46a-dc2534b0db9d
13222024-08-30T19:42:44.528ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_external_dns_ff52340f-525a-4b88-bfc8-1cb0fe9ceba7
13232024-08-30T19:42:44.528ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_crucible_pantry_ff2478ea-0538-402c-8476-a8f41b89a1dd
13242024-08-30T19:42:44.528ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_crucible_e1f7623b-2d1f-4b94-b59f-b88843322bd4
13252024-08-30T19:42:44.528ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_crucible_3c79a17e-c108-4cbb-9ac6-cc88acef9672
13262024-08-30T19:42:44.553ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/fc9e5708-cd50-4950-8336-db011df04e25/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
13272024-08-30T19:42:44.555ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/17afe9d4-47cb-4799-8d3d-ad69badedc16/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
13282024-08-30T19:42:44.557ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 27283580 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:35983 req_id = 551c944c-995d-4e49-a5a5-e1b856a22eb4 response_code = 204 uri = /omicron-zones
13292024-08-30T19:42:44.574ZINFOSledAgent (RSS): Finished setting up services file = sled-agent/src/rack_setup/service.rs:1286
13302024-08-30T19:42:44.574ZINFOSledAgent (RSS): Writing ledger to /pool/int/fc9e5708-cd50-4950-8336-db011df04e25/config/.rss-plan-completed.marker.tmp file = common/src/ledger.rs:199
13312024-08-30T19:42:44.575ZINFOSledAgent (RSS): Writing ledger to /pool/int/17afe9d4-47cb-4799-8d3d-ad69badedc16/config/.rss-plan-completed.marker.tmp file = common/src/ledger.rs:199
13322024-08-30T19:42:44.595ZINFOSledAgent (RSS): Handing off control to Nexus file = sled-agent/src/rack_setup/service.rs:687
13332024-08-30T19:42:44.597ZINFOSledAgent (RSS): Nexus address: [fd00:1122:3344:101::b]:12221 file = sled-agent/src/rack_setup/service.rs:700
13342024-08-30T19:42:44.630ZINFOSledAgent (RSS): rack_network_config: RackNetworkConfigV2 { bfd: [], bgp: [], infra_ip_first: 10.151.1.174, infra_ip_last: 10.151.1.174, ports: [ PortConfigV2 { addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.1.174, width: 24, }, ), vlan_id: None, }, ], autoneg: false, bgp_peers: [], lldp: None, port: "qsfp0", routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), local_pref: None, nexthop: 10.151.1.100, vlan_id: None, }, ], switch: Switch0, uplink_port_fec: None, uplink_port_speed: Speed40G, }, ], rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, } file = sled-agent/src/rack_setup/service.rs:867
13352024-08-30T19:42:44.631ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/27cfef72-d05b-490f-ac92-5c7cad830c2e/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13362024-08-30T19:42:44.779ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/27cfef72-d05b-490f-ac92-5c7cad830c2e/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13372024-08-30T19:42:44.856ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/27cfef72-d05b-490f-ac92-5c7cad830c2e/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13382024-08-30T19:42:45.034ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/27cfef72-d05b-490f-ac92-5c7cad830c2e/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13392024-08-30T19:42:45.274ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/27cfef72-d05b-490f-ac92-5c7cad830c2e/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13402024-08-30T19:42:45.550ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/27cfef72-d05b-490f-ac92-5c7cad830c2e/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13412024-08-30T19:42:45.702ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/27cfef72-d05b-490f-ac92-5c7cad830c2e/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13422024-08-30T19:42:45.877ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/27cfef72-d05b-490f-ac92-5c7cad830c2e/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13432024-08-30T19:42:46.337ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/27cfef72-d05b-490f-ac92-5c7cad830c2e/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13442024-08-30T19:42:46.823ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/27cfef72-d05b-490f-ac92-5c7cad830c2e/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13452024-08-30T19:42:47.290ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/27cfef72-d05b-490f-ac92-5c7cad830c2e/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13462024-08-30T19:42:47.866ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/27cfef72-d05b-490f-ac92-5c7cad830c2e/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13472024-08-30T19:42:48.245ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/27cfef72-d05b-490f-ac92-5c7cad830c2e/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13482024-08-30T19:42:49.467ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/27cfef72-d05b-490f-ac92-5c7cad830c2e/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13492024-08-30T19:42:50.510ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/27cfef72-d05b-490f-ac92-5c7cad830c2e/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13502024-08-30T19:42:51.306ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/27cfef72-d05b-490f-ac92-5c7cad830c2e/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13512024-08-30T19:42:52.803ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/27cfef72-d05b-490f-ac92-5c7cad830c2e/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13522024-08-30T19:42:54.382ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/27cfef72-d05b-490f-ac92-5c7cad830c2e/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13532024-08-30T19:42:56.854ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/27cfef72-d05b-490f-ac92-5c7cad830c2e/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13542024-08-30T19:42:56.983ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 35.80325436s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers): error trying to connect: tcp connect error: Connection refused (os error 146)" file = oximeter/producer/src/lib.rs:424 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be
13552024-08-30T19:42:58.658ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/27cfef72-d05b-490f-ac92-5c7cad830c2e/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13562024-08-30T19:43:02.890ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "2e847fbb-2450-46d7-ad16-8ce29dc0a9c8", "content-length": "173", "date": "Fri, 30 Aug 2024 19:43:02 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"289c9d10-fa69-4638-aac7-3cfc6b4e23be\"", request_id: "2e847fbb-2450-46d7-ad16-8ce29dc0a9c8" } file = sled-agent/src/rack_setup/service.rs:929
13572024-08-30T19:43:08.940ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "2b71dd4e-7453-4da0-972e-f970a3233ba8", "content-length": "173", "date": "Fri, 30 Aug 2024 19:43:08 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"289c9d10-fa69-4638-aac7-3cfc6b4e23be\"", request_id: "2b71dd4e-7453-4da0-972e-f970a3233ba8" } file = sled-agent/src/rack_setup/service.rs:929
13582024-08-30T19:43:11.668ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:58676
13592024-08-30T19:43:11.668ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 44 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:58676 req_id = 50c2f949-8f55-4b58-8104-41fae2ff55c3 response_code = 200 uri = /baseboard
13602024-08-30T19:43:13.318ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "c6d53823-6eb6-44c5-a5b4-b7b47d89ec51", "content-length": "173", "date": "Fri, 30 Aug 2024 19:43:13 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"289c9d10-fa69-4638-aac7-3cfc6b4e23be\"", request_id: "c6d53823-6eb6-44c5-a5b4-b7b47d89ec51" } file = sled-agent/src/rack_setup/service.rs:929
13612024-08-30T19:43:17.344ZINFOSledAgent (NexusNotifierTask): Successfully put SledAgentInfo to nexus file = sled-agent/src/nexus.rs:544 new_generation = 1 old_generation = 1 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be
13622024-08-30T19:43:17.426ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:49207
13632024-08-30T19:43:17.429ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
13642024-08-30T19:43:17.515ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 88699 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:49207 req_id = 9c81b7f4-b7a3-4031-a917-a3c27c625e4d response_code = 200 uri = /inventory
13652024-08-30T19:43:17.515ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
13662024-08-30T19:43:17.515ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/fc9e5708-cd50-4950-8336-db011df04e25/config/omicron-zones.json file = common/src/ledger.rs:177
13672024-08-30T19:43:17.518ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/17afe9d4-47cb-4799-8d3d-ad69badedc16/config/omicron-zones.json file = common/src/ledger.rs:177
13682024-08-30T19:43:17.519ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3357 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:49207 req_id = 72a109e3-ca84-42f8-a54c-634d519b80cc response_code = 200 uri = /omicron-zones
13692024-08-30T19:43:18.853ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:49340
13702024-08-30T19:43:18.856ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 148fa4d0-99ac-4f4c-b957-d26cc8730745, kind: Service { id: b5636b10-76ec-4aa4-8bd4-1f95780813df }, name: Name("external-dns-b5636b10-76ec-4aa4-8bd4-1f95780813df"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 151, 107])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e6ac84e1-d9f1-4c03-8dd9-78d5c3b887de, kind: Service { id: ff52340f-525a-4b88-bfc8-1cb0fe9ceba7 }, name: Name("external-dns-ff52340f-525a-4b88-bfc8-1cb0fe9ceba7"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 231])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 2495ea70-4560-412b-9ebd-45a2bc27810a, kind: Service { id: d2343105-862a-478c-b746-fd8bb78f9a96 }, name: Name("nexus-d2343105-862a-478c-b746-fd8bb78f9a96"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 101])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 3e58b382-51e7-46a7-8c87-6ca3a6547a20, kind: Service { id: 1d96d476-acdb-4c44-a486-29569b3bf41e }, name: Name("nexus-1d96d476-acdb-4c44-a486-29569b3bf41e"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 98])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 82aefc08-c1a4-48ce-8225-d8f4409ab816, kind: Service { id: ba4aef74-7b33-4ded-a46a-dc2534b0db9d }, name: Name("nexus-ba4aef74-7b33-4ded-a46a-dc2534b0db9d"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 158, 211])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
13712024-08-30T19:43:18.856ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
13722024-08-30T19:43:18.865ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
13732024-08-30T19:43:18.865ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
13742024-08-30T19:43:18.865ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
13752024-08-30T19:43:18.880ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
13762024-08-30T19:43:18.880ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
13772024-08-30T19:43:18.880ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 26688 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:49340 req_id = 1ba881b3-af35-4922-911a-d5a9392d28fc response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
13782024-08-30T19:43:18.881ZINFOSledAgent (RSS): Handoff to Nexus is complete file = sled-agent/src/rack_setup/service.rs:939
13792024-08-30T19:43:19.197ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:36842
13802024-08-30T19:43:19.197ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
13812024-08-30T19:43:19.284ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 86874 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:36842 req_id = 239c2bcb-221d-4ea1-bd73-e79741b7f7c5 response_code = 200 uri = /inventory
13822024-08-30T19:43:19.285ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
13832024-08-30T19:43:19.285ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/fc9e5708-cd50-4950-8336-db011df04e25/config/omicron-zones.json file = common/src/ledger.rs:177
13842024-08-30T19:43:19.285ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/17afe9d4-47cb-4799-8d3d-ad69badedc16/config/omicron-zones.json file = common/src/ledger.rs:177
13852024-08-30T19:43:19.286ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1229 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:36842 req_id = 95f86352-41f2-4877-8ec0-cbd16348cc91 response_code = 200 uri = /omicron-zones
13862024-08-30T19:43:19.669ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:52355
13872024-08-30T19:43:19.669ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 148fa4d0-99ac-4f4c-b957-d26cc8730745, kind: Service { id: b5636b10-76ec-4aa4-8bd4-1f95780813df }, name: Name("external-dns-b5636b10-76ec-4aa4-8bd4-1f95780813df"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 151, 107])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e6ac84e1-d9f1-4c03-8dd9-78d5c3b887de, kind: Service { id: ff52340f-525a-4b88-bfc8-1cb0fe9ceba7 }, name: Name("external-dns-ff52340f-525a-4b88-bfc8-1cb0fe9ceba7"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 231])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 2495ea70-4560-412b-9ebd-45a2bc27810a, kind: Service { id: d2343105-862a-478c-b746-fd8bb78f9a96 }, name: Name("nexus-d2343105-862a-478c-b746-fd8bb78f9a96"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 101])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 3e58b382-51e7-46a7-8c87-6ca3a6547a20, kind: Service { id: 1d96d476-acdb-4c44-a486-29569b3bf41e }, name: Name("nexus-1d96d476-acdb-4c44-a486-29569b3bf41e"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 98])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 82aefc08-c1a4-48ce-8225-d8f4409ab816, kind: Service { id: ba4aef74-7b33-4ded-a46a-dc2534b0db9d }, name: Name("nexus-ba4aef74-7b33-4ded-a46a-dc2534b0db9d"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 158, 211])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
13882024-08-30T19:43:19.669ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
13892024-08-30T19:43:19.670ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
13902024-08-30T19:43:19.670ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
13912024-08-30T19:43:19.670ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
13922024-08-30T19:43:19.670ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
13932024-08-30T19:43:19.670ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
13942024-08-30T19:43:19.670ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 342 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:52355 req_id = b0d6464a-6ebd-4cd8-99a4-3f08b4b11190 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
13952024-08-30T19:43:19.887ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:44734
13962024-08-30T19:43:19.888ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 148fa4d0-99ac-4f4c-b957-d26cc8730745, kind: Service { id: b5636b10-76ec-4aa4-8bd4-1f95780813df }, name: Name("external-dns-b5636b10-76ec-4aa4-8bd4-1f95780813df"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 151, 107])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e6ac84e1-d9f1-4c03-8dd9-78d5c3b887de, kind: Service { id: ff52340f-525a-4b88-bfc8-1cb0fe9ceba7 }, name: Name("external-dns-ff52340f-525a-4b88-bfc8-1cb0fe9ceba7"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 231])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 2495ea70-4560-412b-9ebd-45a2bc27810a, kind: Service { id: d2343105-862a-478c-b746-fd8bb78f9a96 }, name: Name("nexus-d2343105-862a-478c-b746-fd8bb78f9a96"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 101])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 3e58b382-51e7-46a7-8c87-6ca3a6547a20, kind: Service { id: 1d96d476-acdb-4c44-a486-29569b3bf41e }, name: Name("nexus-1d96d476-acdb-4c44-a486-29569b3bf41e"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 98])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 82aefc08-c1a4-48ce-8225-d8f4409ab816, kind: Service { id: ba4aef74-7b33-4ded-a46a-dc2534b0db9d }, name: Name("nexus-ba4aef74-7b33-4ded-a46a-dc2534b0db9d"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 158, 211])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
13972024-08-30T19:43:19.888ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
13982024-08-30T19:43:19.888ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
13992024-08-30T19:43:19.888ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
14002024-08-30T19:43:19.888ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
14012024-08-30T19:43:19.888ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
14022024-08-30T19:43:19.888ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
14032024-08-30T19:43:19.888ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 323 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:44734 req_id = 353c44ca-f264-490f-90fd-934f8b3ca0e3 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
14042024-08-30T19:43:20.799ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:51381
14052024-08-30T19:43:20.799ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 148fa4d0-99ac-4f4c-b957-d26cc8730745, kind: Service { id: b5636b10-76ec-4aa4-8bd4-1f95780813df }, name: Name("external-dns-b5636b10-76ec-4aa4-8bd4-1f95780813df"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 151, 107])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e6ac84e1-d9f1-4c03-8dd9-78d5c3b887de, kind: Service { id: ff52340f-525a-4b88-bfc8-1cb0fe9ceba7 }, name: Name("external-dns-ff52340f-525a-4b88-bfc8-1cb0fe9ceba7"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 231])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 2495ea70-4560-412b-9ebd-45a2bc27810a, kind: Service { id: d2343105-862a-478c-b746-fd8bb78f9a96 }, name: Name("nexus-d2343105-862a-478c-b746-fd8bb78f9a96"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 101])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 3e58b382-51e7-46a7-8c87-6ca3a6547a20, kind: Service { id: 1d96d476-acdb-4c44-a486-29569b3bf41e }, name: Name("nexus-1d96d476-acdb-4c44-a486-29569b3bf41e"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 98])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 82aefc08-c1a4-48ce-8225-d8f4409ab816, kind: Service { id: ba4aef74-7b33-4ded-a46a-dc2534b0db9d }, name: Name("nexus-ba4aef74-7b33-4ded-a46a-dc2534b0db9d"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 158, 211])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
14062024-08-30T19:43:20.799ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
14072024-08-30T19:43:20.799ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
14082024-08-30T19:43:20.799ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
14092024-08-30T19:43:20.799ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
14102024-08-30T19:43:20.799ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
14112024-08-30T19:43:20.799ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
14122024-08-30T19:43:20.799ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 441 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:51381 req_id = 954e6510-a5de-4cd6-a068-a64af8198aeb response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
14132024-08-30T19:43:27.370ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:37187
14142024-08-30T19:43:27.379ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8749 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:37187 req_id = 5ac7bcf7-5edd-462f-ae58-4a7705b6700e response_code = 200 uri = /vpc-routes
14152024-08-30T19:43:27.461ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8736 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:37187 req_id = 47f10fb8-0887-41f8-a6f4-d0ba658c4417 response_code = 204 uri = /vpc-routes
14162024-08-30T19:43:31.718ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:39182
14172024-08-30T19:43:31.719ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1053 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:39182 req_id = 8a648c22-e1c1-4009-b416-6bc934ad5ea4 response_code = 200 uri = /vpc-routes
14182024-08-30T19:43:31.731ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 118 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:39182 req_id = 4a06734b-58f0-40a2-86bb-1b1cb78b4ce7 response_code = 204 uri = /vpc-routes
14192024-08-30T19:43:31.837ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:61603
14202024-08-30T19:43:31.837ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 548 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:61603 req_id = 55c1103e-fb20-484f-9adf-f1ac19971704 response_code = 200 uri = /v2p
14212024-08-30T19:43:31.880ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:32779
14222024-08-30T19:43:31.881ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
14232024-08-30T19:43:32.009ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
14242024-08-30T19:43:32.076ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 195742 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:32779 req_id = 5399859b-dba8-4767-a395-a561038d4a9a response_code = 204 uri = /switch-ports
14252024-08-30T19:43:32.076ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:62824
14262024-08-30T19:43:32.076ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:47015
14272024-08-30T19:43:32.076ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:40442
14282024-08-30T19:43:32.076ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:54312
14292024-08-30T19:43:32.077ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 903 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:47015 req_id = 39243c01-572b-49b2-a4d7-b8423e67e2fc response_code = 200 uri = /vpc-routes
14302024-08-30T19:43:32.077ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 889 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:40442 req_id = 8a756d1d-2126-4da8-8ec4-87c5ea79243a response_code = 200 uri = /vpc-routes
14312024-08-30T19:43:32.077ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
14322024-08-30T19:43:32.077ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 882 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:54312 req_id = 0918e11e-479a-4e43-83f2-d6b769d7898e response_code = 200 uri = /v2p
14332024-08-30T19:43:32.084ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 7924 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:32779 req_id = 6e4236a6-f3a7-4750-9b20-db874a7ddb91 response_code = 200 uri = /network-bootstore-config
14342024-08-30T19:43:32.108ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:49852
14352024-08-30T19:43:32.109ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 415 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:49852 req_id = b4364c5f-a62f-46b9-840b-2147d85ccf2f response_code = 200 uri = /v2p
14362024-08-30T19:43:32.113ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 210 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:47015 req_id = 9507b931-3604-445d-823c-1e189cd6e25b response_code = 204 uri = /vpc-routes
14372024-08-30T19:43:32.113ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 75 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:40442 req_id = 47cfb89e-b8e4-4950-b257-eb4a85b3383c response_code = 204 uri = /vpc-routes
14382024-08-30T19:43:32.120ZINFOSledAgent (bootstore): Attempting to update network config with generation: 2, current_generation: 1 file = bootstore/src/schemes/v0/peer.rs:564 peer_id = pc-buskin-i86pc
14392024-08-30T19:43:32.121ZINFOSledAgent (bootstore): Writing ledger to /pool/int/fc9e5708-cd50-4950-8336-db011df04e25/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-buskin-i86pc
14402024-08-30T19:43:32.122ZINFOSledAgent (bootstore): Writing ledger to /pool/int/17afe9d4-47cb-4799-8d3d-ad69badedc16/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-buskin-i86pc
14412024-08-30T19:43:32.123ZINFOSledAgent (bootstore): Broadcasting network config with generation 2 file = bootstore/src/schemes/v0/peer.rs:648 peer_id = pc-buskin-i86pc
14422024-08-30T19:43:32.123ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3232 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:32779 req_id = 48811b7e-cdab-4179-8e31-5452a03da584 response_code = 204 uri = /network-bootstore-config
14432024-08-30T19:43:32.125ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
14442024-08-30T19:43:32.192ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 115371 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:62824 req_id = e3438e3e-eca9-4264-b1c1-2f0416502569 response_code = 204 uri = /switch-ports
14452024-08-30T19:43:32.193ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 713 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:62824 req_id = 15cda0fc-a7a3-4911-abb8-c34278480a1f response_code = 200 uri = /network-bootstore-config
14462024-08-30T19:43:32.358ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:45752
14472024-08-30T19:43:32.358ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
14482024-08-30T19:43:32.394ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
14492024-08-30T19:43:32.444ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 85639 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:45752 req_id = f6aadbde-dafd-450c-96e9-f4a24fe2908f response_code = 204 uri = /switch-ports
14502024-08-30T19:43:32.444ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 131 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:45752 req_id = 97f446a9-8a82-4c5e-9fa9-848f0eb101dc response_code = 200 uri = /network-bootstore-config
14512024-08-30T19:43:41.700ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:44846
14522024-08-30T19:43:41.700ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 634 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:44846 req_id = fb697fe6-ae75-4fac-8ac8-cc418d405217 response_code = 200 uri = /baseboard
14532024-08-30T19:44:01.630ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:43251
14542024-08-30T19:44:01.631ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 97 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:43251 req_id = 549d1a0b-df25-4746-b55f-746cc1f4edcd response_code = 200 uri = /vpc-routes
14552024-08-30T19:44:01.655ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 199 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:43251 req_id = 106eb1cf-02d7-4d02-a9d2-a521e9975e26 response_code = 204 uri = /vpc-routes
14562024-08-30T19:44:01.759ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:35554
14572024-08-30T19:44:01.760ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 913 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:35554 req_id = 4da151b2-8c2a-4f62-834c-f8c22aad203d response_code = 200 uri = /vpc-routes
14582024-08-30T19:44:01.783ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 131 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:35554 req_id = 56fe8886-4058-4bb3-87f3-28d5b59029d2 response_code = 204 uri = /vpc-routes
14592024-08-30T19:44:02.031ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:40248
14602024-08-30T19:44:02.032ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 173 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:40248 req_id = 9b05c07c-0276-43f0-833f-e9872e624ac9 response_code = 200 uri = /v2p
14612024-08-30T19:44:02.100ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:60063
14622024-08-30T19:44:02.101ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 834 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:60063 req_id = 12d1f905-3587-412a-b413-6d110ca8257f response_code = 200 uri = /v2p
14632024-08-30T19:44:02.132ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:62789
14642024-08-30T19:44:02.132ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
14652024-08-30T19:44:02.168ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
14662024-08-30T19:44:02.229ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 97264 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:62789 req_id = fe845ef5-db37-4739-a00e-40128ce90d5d response_code = 204 uri = /switch-ports
14672024-08-30T19:44:02.229ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:40677
14682024-08-30T19:44:02.230ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 755 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:40677 req_id = b655a8a7-779e-46a8-add5-555bc7d9fc13 response_code = 200 uri = /vpc-routes
14692024-08-30T19:44:02.230ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 90 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:62789 req_id = e3c2f902-7ec0-41f9-adf5-5c0b8ac0412a response_code = 200 uri = /network-bootstore-config
14702024-08-30T19:44:02.240ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 124 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:40677 req_id = c032d768-6c4e-4a8a-bdc3-4a3bc4adea65 response_code = 204 uri = /vpc-routes
14712024-08-30T19:44:02.306ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:48587
14722024-08-30T19:44:02.307ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
14732024-08-30T19:44:02.343ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
14742024-08-30T19:44:02.406ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 99565 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:48587 req_id = 9e699934-4b55-41a5-abcd-a78e57aa075a response_code = 204 uri = /switch-ports
14752024-08-30T19:44:02.406ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:58227
14762024-08-30T19:44:02.406ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:63899
14772024-08-30T19:44:02.406ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
14782024-08-30T19:44:02.412ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 5886 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:63899 req_id = 3f94909e-4f42-4579-bca5-2e0a4819439d response_code = 200 uri = /v2p
14792024-08-30T19:44:02.412ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 6051 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:48587 req_id = f585194e-4e6f-40f4-8328-ff07a2088327 response_code = 200 uri = /network-bootstore-config
14802024-08-30T19:44:02.444ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
14812024-08-30T19:44:02.502ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 95730 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:58227 req_id = 832ad6b4-8b63-489f-8d57-b77884de9db6 response_code = 204 uri = /switch-ports
14822024-08-30T19:44:02.503ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 857 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:58227 req_id = 71d70530-ef68-402f-83d9-3d35ba6c46eb response_code = 200 uri = /network-bootstore-config
14832024-08-30T19:44:02.922ZINFOSledAgent (dropshot): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:48378 remote_addr = [fd00:1122:3344:101::d]:42071 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be
14842024-08-30T19:44:02.953ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 30951 local_addr = [fd00:1122:3344:101::1]:48378 method = GET remote_addr = [fd00:1122:3344:101::d]:42071 req_id = b294053a-769e-4329-b2ef-f475a2d9a51b response_code = 200 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be uri = /289c9d10-fa69-4638-aac7-3cfc6b4e23be
14852024-08-30T19:44:11.742ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:59633
14862024-08-30T19:44:11.742ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 55 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:59633 req_id = 81bfa26c-db3e-4db1-8cf4-a07d83ced83b response_code = 200 uri = /baseboard
14872024-08-30T19:44:31.751ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:37012
14882024-08-30T19:44:31.752ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 791 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:37012 req_id = bd9eae96-ac73-468e-bdb1-f0de3a05a267 response_code = 200 uri = /vpc-routes
14892024-08-30T19:44:31.822ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:41336
14902024-08-30T19:44:31.822ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 155 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:41336 req_id = 3a589b17-68f9-44a3-b12e-a48144a914f6 response_code = 200 uri = /v2p
14912024-08-30T19:44:31.878ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:54862
14922024-08-30T19:44:31.878ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 76 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:54862 req_id = 38869561-235d-4e69-83a3-db943f28435a response_code = 200 uri = /vpc-routes
14932024-08-30T19:44:31.938ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 132 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:37012 req_id = 453f09c9-4a4e-478c-a449-ff31020ee205 response_code = 204 uri = /vpc-routes
14942024-08-30T19:44:31.952ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:64857
14952024-08-30T19:44:31.952ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 742 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:64857 req_id = 7144d732-52a1-41d2-8918-3d8972f60c80 response_code = 200 uri = /vpc-routes
14962024-08-30T19:44:32.008ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:39549
14972024-08-30T19:44:32.008ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 151 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:39549 req_id = f6b13cd1-25bb-402b-a59a-26c2ba834923 response_code = 200 uri = /v2p
14982024-08-30T19:44:32.024ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1658 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:54862 req_id = acb6f049-5eeb-40c1-aac7-f2f23c7faedb response_code = 204 uri = /vpc-routes
14992024-08-30T19:44:32.040ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:59152
15002024-08-30T19:44:32.040ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
15012024-08-30T19:44:32.084ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
15022024-08-30T19:44:32.142ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 101703 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:59152 req_id = eeb31337-b8c7-4a93-9676-f0c897a4b6e5 response_code = 204 uri = /switch-ports
15032024-08-30T19:44:32.142ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:63228
15042024-08-30T19:44:32.142ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:38041
15052024-08-30T19:44:32.142ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:48347
15062024-08-30T19:44:32.142ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 364 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:59152 req_id = 89f35935-a3c4-4e37-8601-1dba810766e9 response_code = 200 uri = /network-bootstore-config
15072024-08-30T19:44:32.142ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
15082024-08-30T19:44:32.142ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 415 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:64857 req_id = 164dc07b-393f-4d1f-846e-1bff040df3ab response_code = 204 uri = /vpc-routes
15092024-08-30T19:44:32.142ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 257 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:38041 req_id = 5bfdfd86-2f4a-4065-9f5d-db134743c018 response_code = 200 uri = /v2p
15102024-08-30T19:44:32.185ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
15112024-08-30T19:44:32.242ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
15122024-08-30T19:44:32.242ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 100105 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:48347 req_id = d58830fd-8c90-4ed7-b2fd-629d511fcd54 response_code = 204 uri = /switch-ports
15132024-08-30T19:44:32.246ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 525 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:48347 req_id = 51dfb21e-f19f-4874-a587-a2494b76284d response_code = 200 uri = /network-bootstore-config
15142024-08-30T19:44:32.278ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
15152024-08-30T19:44:32.333ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 190860 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:63228 req_id = 0966a1fd-02f3-4941-9432-d92c0555cec2 response_code = 204 uri = /switch-ports
15162024-08-30T19:44:32.333ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 127 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:63228 req_id = b83b23fe-83a3-4f35-95fc-206ae90fbab6 response_code = 200 uri = /network-bootstore-config
15172024-08-30T19:44:32.929ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 6798 local_addr = [fd00:1122:3344:101::1]:48378 method = GET remote_addr = [fd00:1122:3344:101::d]:42071 req_id = 65f1fd63-25ab-4571-8449-ca43b3647f35 response_code = 200 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be uri = /289c9d10-fa69-4638-aac7-3cfc6b4e23be
15182024-08-30T19:44:41.776ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:33132
15192024-08-30T19:44:41.776ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 107 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:33132 req_id = f1dbeb0f-c55d-4e55-82a6-94ba0fb12941 response_code = 200 uri = /baseboard
15202024-08-30T19:45:01.875ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51821
15212024-08-30T19:45:01.876ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 136 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:51821 req_id = 4f5e9a64-486f-4f4a-b1dc-2c29d269f38d response_code = 200 uri = /vpc-routes
15222024-08-30T19:45:01.969ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:59972
15232024-08-30T19:45:01.970ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 176 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:59972 req_id = 6659a13b-9a15-4ace-bded-225370bf6c50 response_code = 200 uri = /v2p
15242024-08-30T19:45:01.982ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 139 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:51821 req_id = e5e8a2b8-537c-46d0-877e-d4fab3dd7fcd response_code = 204 uri = /vpc-routes
15252024-08-30T19:45:02.007ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:39730
15262024-08-30T19:45:02.007ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 47 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:39730 req_id = c87a665c-6bd4-4366-97bc-8420f1b8fc51 response_code = 200 uri = /vpc-routes
15272024-08-30T19:45:02.129ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:48428
15282024-08-30T19:45:02.129ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
15292024-08-30T19:45:02.201ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
15302024-08-30T19:45:02.311ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 181758 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:48428 req_id = fd338816-67ff-41e1-9c5c-40aa66424984 response_code = 204 uri = /switch-ports
15312024-08-30T19:45:02.311ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:44850
15322024-08-30T19:45:02.312ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:56476
15332024-08-30T19:45:02.312ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 294 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:44850 req_id = 82683795-25f3-4ba2-9a21-1f6a6df9e040 response_code = 200 uri = /v2p
15342024-08-30T19:45:02.312ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 273 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:48428 req_id = 81638a8e-5db4-4aa8-ab20-5db12410bb16 response_code = 200 uri = /network-bootstore-config
15352024-08-30T19:45:02.312ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 355 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:39730 req_id = a5c304a3-685a-4c03-9d3e-863781700243 response_code = 204 uri = /vpc-routes
15362024-08-30T19:45:02.313ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1510 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:56476 req_id = fa3a745b-ab43-4294-8d92-58f890576f35 response_code = 200 uri = /vpc-routes
15372024-08-30T19:45:02.372ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:34941
15382024-08-30T19:45:02.372ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
15392024-08-30T19:45:02.431ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
15402024-08-30T19:45:02.528ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 156216 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:34941 req_id = 3305201e-f9ad-4447-8f28-c8bf658b37cd response_code = 204 uri = /switch-ports
15412024-08-30T19:45:02.528ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:48734
15422024-08-30T19:45:02.529ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 306 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:56476 req_id = ca586a9a-0549-4a6a-9572-f2121df2b369 response_code = 204 uri = /vpc-routes
15432024-08-30T19:45:02.529ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 297 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:48734 req_id = 02724eeb-1ac1-4b19-92ea-f90d3a52e5b1 response_code = 200 uri = /v2p
15442024-08-30T19:45:02.529ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 379 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:34941 req_id = 3a8c1f57-cac7-40a0-b977-d161680d0f45 response_code = 200 uri = /network-bootstore-config
15452024-08-30T19:45:02.549ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:53610
15462024-08-30T19:45:02.549ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
15472024-08-30T19:45:02.601ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
15482024-08-30T19:45:02.677ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 128564 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:53610 req_id = a541816f-62dd-4f2f-800c-fdc7bb37352f response_code = 204 uri = /switch-ports
15492024-08-30T19:45:02.678ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 155 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:53610 req_id = 97ad8c55-ea00-4391-980f-026f5f4bbd8e response_code = 200 uri = /network-bootstore-config
15502024-08-30T19:45:02.929ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 6731 local_addr = [fd00:1122:3344:101::1]:48378 method = GET remote_addr = [fd00:1122:3344:101::d]:42071 req_id = 6b7cc12b-7a17-48e3-b84a-ddbb6b213f84 response_code = 200 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be uri = /289c9d10-fa69-4638-aac7-3cfc6b4e23be
15512024-08-30T19:45:11.809ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:34017
15522024-08-30T19:45:11.809ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 112 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:34017 req_id = 7a34da5e-d72d-46bb-a7ba-9f3226380126 response_code = 200 uri = /baseboard
15532024-08-30T19:45:31.687ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:52080
15542024-08-30T19:45:31.687ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 85 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:52080 req_id = 4a3f34d2-ab78-471f-959e-297345b1b2f8 response_code = 200 uri = /vpc-routes
15552024-08-30T19:45:31.777ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:46952
15562024-08-30T19:45:31.780ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2046 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:46952 req_id = 4362b150-8b75-4dbf-89c8-bb31c79dcd2b response_code = 200 uri = /v2p
15572024-08-30T19:45:31.801ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 148 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:52080 req_id = f5c0880c-cc23-4493-979b-60a4553220cd response_code = 204 uri = /vpc-routes
15582024-08-30T19:45:31.849ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:45486
15592024-08-30T19:45:31.849ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 60 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:45486 req_id = 97461ee5-7fd0-444d-8171-875a1ea4449d response_code = 200 uri = /vpc-routes
15602024-08-30T19:45:31.953ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:56540
15612024-08-30T19:45:31.953ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
15622024-08-30T19:45:31.990ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
15632024-08-30T19:45:32.047ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 94183 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:56540 req_id = 2688ff24-d623-49d6-b5f6-c2b7841ffdc9 response_code = 204 uri = /switch-ports
15642024-08-30T19:45:32.047ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:57785
15652024-08-30T19:45:32.047ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:44412
15662024-08-30T19:45:32.048ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
15672024-08-30T19:45:32.048ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 905 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:57785 req_id = 6c31b0a2-28ee-4d43-835f-78c7c8af7dc2 response_code = 200 uri = /v2p
15682024-08-30T19:45:32.048ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 921 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:56540 req_id = 364c71c0-cb2a-4cfc-a8c5-c69dfa5193bf response_code = 200 uri = /network-bootstore-config
15692024-08-30T19:45:32.052ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 4651 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:45486 req_id = eb99068c-f541-4cab-adea-5bfd39e330b5 response_code = 204 uri = /vpc-routes
15702024-08-30T19:45:32.074ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:54873
15712024-08-30T19:45:32.075ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 173 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:54873 req_id = 72e01f4f-3a86-466e-8560-df6bbe4ebf29 response_code = 200 uri = /vpc-routes
15722024-08-30T19:45:32.083ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
15732024-08-30T19:45:32.087ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 562 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:54873 req_id = 9088f0e2-79d8-4026-80b3-dd08a197e23b response_code = 204 uri = /vpc-routes
15742024-08-30T19:45:32.138ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 90906 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:44412 req_id = 5723d84c-6fdf-4274-865e-fa0bcbae092a response_code = 204 uri = /switch-ports
15752024-08-30T19:45:32.139ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 189 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:44412 req_id = 646b4250-e8cd-4948-83b1-43834563679b response_code = 200 uri = /network-bootstore-config
15762024-08-30T19:45:32.178ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:64847
15772024-08-30T19:45:32.179ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 744 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:64847 req_id = 4c579798-6099-4f9b-bb40-ef9958a6d259 response_code = 200 uri = /v2p
15782024-08-30T19:45:32.190ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:58741
15792024-08-30T19:45:32.190ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
15802024-08-30T19:45:32.224ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
15812024-08-30T19:45:32.275ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 84560 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:58741 req_id = bab1acd3-7dc7-4944-b05b-040373281ae5 response_code = 204 uri = /switch-ports
15822024-08-30T19:45:32.275ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 113 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:58741 req_id = d22ea5a5-7d40-4d2e-9a80-6a74d5e00d4d response_code = 200 uri = /network-bootstore-config
15832024-08-30T19:45:32.929ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 6914 local_addr = [fd00:1122:3344:101::1]:48378 method = GET remote_addr = [fd00:1122:3344:101::d]:42071 req_id = dc0d5147-e81d-43be-969f-1a29822471e6 response_code = 200 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be uri = /289c9d10-fa69-4638-aac7-3cfc6b4e23be
15842024-08-30T19:45:41.843ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:34881
15852024-08-30T19:45:41.843ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 82 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:34881 req_id = a0449d4a-9337-451b-9a99-2c71c102ee29 response_code = 200 uri = /baseboard
15862024-08-30T19:45:50.753ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
15872024-08-30T19:46:01.794ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:37963
15882024-08-30T19:46:01.794ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 132 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:37963 req_id = b0cc9e0b-271f-44e2-b3af-721086a74210 response_code = 200 uri = /vpc-routes
15892024-08-30T19:46:01.882ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:65509
15902024-08-30T19:46:01.883ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 759 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:65509 req_id = f5578d08-3008-4623-8ee2-6a71edd1a8ec response_code = 200 uri = /v2p
15912024-08-30T19:46:01.892ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 143 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:37963 req_id = 67f85d2f-8cc0-455e-a376-66aa3999b53e response_code = 204 uri = /vpc-routes
15922024-08-30T19:46:01.967ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:50067
15932024-08-30T19:46:01.967ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 289 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:50067 req_id = 7de4ed21-2f2d-4a27-b26d-746876adbd02 response_code = 200 uri = /vpc-routes
15942024-08-30T19:46:01.989ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1409 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:50067 req_id = 80a19ebe-6b86-4109-9cfe-a3e006279f88 response_code = 204 uri = /vpc-routes
15952024-08-30T19:46:02.147ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:38138
15962024-08-30T19:46:02.148ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 933 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:38138 req_id = ef59c290-508c-45bb-b9d1-a0e3e8a48316 response_code = 200 uri = /vpc-routes
15972024-08-30T19:46:02.214ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:42770
15982024-08-30T19:46:02.215ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
15992024-08-30T19:46:02.256ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
16002024-08-30T19:46:02.317ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 102802 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:42770 req_id = 0a6d41fb-de60-4dd3-a75c-e103b7ec6a2d response_code = 204 uri = /switch-ports
16012024-08-30T19:46:02.317ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:62878
16022024-08-30T19:46:02.317ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:39343
16032024-08-30T19:46:02.317ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:59207
16042024-08-30T19:46:02.318ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
16052024-08-30T19:46:02.318ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 951 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:62878 req_id = c45cf39b-35cb-434d-9cd3-c9a4b562306d response_code = 200 uri = /v2p
16062024-08-30T19:46:02.318ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1090 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:42770 req_id = 08f2c7ed-bb4d-4d27-a902-61240913b923 response_code = 200 uri = /network-bootstore-config
16072024-08-30T19:46:02.329ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 11352 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:39343 req_id = d0f61799-7700-49c6-a44f-1281366bdb05 response_code = 200 uri = /v2p
16082024-08-30T19:46:02.329ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 11592 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:38138 req_id = cc28245a-59f8-45e4-a168-eaf3b3d87916 response_code = 204 uri = /vpc-routes
16092024-08-30T19:46:02.355ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
16102024-08-30T19:46:02.405ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 87728 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:59207 req_id = 1961bf71-3509-47bc-b079-7e994c11d790 response_code = 204 uri = /switch-ports
16112024-08-30T19:46:02.405ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 146 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:59207 req_id = ede3456a-2e6d-40f7-8967-6499f71aee40 response_code = 200 uri = /network-bootstore-config
16122024-08-30T19:46:02.407ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:52968
16132024-08-30T19:46:02.407ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
16142024-08-30T19:46:02.441ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
16152024-08-30T19:46:02.494ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 87057 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:52968 req_id = f7875d0e-5ff6-49e2-97e1-169fd6439b41 response_code = 204 uri = /switch-ports
16162024-08-30T19:46:02.494ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 142 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:52968 req_id = 1b0ba706-8843-449a-810c-e488d4fcfcc0 response_code = 200 uri = /network-bootstore-config
16172024-08-30T19:46:02.937ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 5163 local_addr = [fd00:1122:3344:101::1]:48378 method = GET remote_addr = [fd00:1122:3344:101::d]:42071 req_id = bb947aea-561a-4451-9bb7-c0bf616fb1db response_code = 200 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be uri = /289c9d10-fa69-4638-aac7-3cfc6b4e23be
16182024-08-30T19:46:11.880ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:41986
16192024-08-30T19:46:11.881ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 83 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:41986 req_id = 7f0cfc8d-afdc-4b10-aad3-0fd4ab9d53e3 response_code = 200 uri = /baseboard
16202024-08-30T19:46:31.703ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:36104
16212024-08-30T19:46:31.703ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 95 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:36104 req_id = 1b2def8b-a722-430b-bde7-d9e599a6b8b6 response_code = 200 uri = /vpc-routes
16222024-08-30T19:46:31.705ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:45189
16232024-08-30T19:46:31.705ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 173 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:45189 req_id = 46d4908c-0659-4beb-ac41-2daacda91608 response_code = 200 uri = /v2p
16242024-08-30T19:46:31.715ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 134 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:36104 req_id = 6a57fdba-a440-40de-ad2e-b7ca88cf078f response_code = 204 uri = /vpc-routes
16252024-08-30T19:46:31.807ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:33309
16262024-08-30T19:46:31.807ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
16272024-08-30T19:46:31.856ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
16282024-08-30T19:46:31.921ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 113570 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:33309 req_id = f9395817-9257-4022-b002-7b3efd02bde5 response_code = 204 uri = /switch-ports
16292024-08-30T19:46:31.921ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 110 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:33309 req_id = 6c0b0e85-4808-406c-b21c-b00df55c6120 response_code = 200 uri = /network-bootstore-config
16302024-08-30T19:46:31.937ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:62748
16312024-08-30T19:46:31.938ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 960 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:62748 req_id = acb043cd-da09-4a5e-add5-e4aa3885932a response_code = 200 uri = /vpc-routes
16322024-08-30T19:46:31.951ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 132 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:62748 req_id = 59b25884-2375-4c68-b162-28b6f7c7747f response_code = 204 uri = /vpc-routes
16332024-08-30T19:46:31.994ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:33406
16342024-08-30T19:46:31.994ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 103 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:33406 req_id = 0849f86a-7ef4-4ee1-83c3-07d01666ca6e response_code = 200 uri = /vpc-routes
16352024-08-30T19:46:32.004ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 145 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:33406 req_id = c1490272-19ac-4f99-bee5-af44480dbe73 response_code = 204 uri = /vpc-routes
16362024-08-30T19:46:32.010ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:56328
16372024-08-30T19:46:32.011ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 892 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:56328 req_id = bb038d43-f0d0-43a4-b1e7-c7319fc220b5 response_code = 200 uri = /v2p
16382024-08-30T19:46:32.082ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:41147
16392024-08-30T19:46:32.082ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
16402024-08-30T19:46:32.123ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
16412024-08-30T19:46:32.178ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 95666 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:41147 req_id = ef6d4ad7-582e-486d-b725-0c3f36fd11d1 response_code = 204 uri = /switch-ports
16422024-08-30T19:46:32.178ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:62396
16432024-08-30T19:46:32.178ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:35170
16442024-08-30T19:46:32.178ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
16452024-08-30T19:46:32.184ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 5530 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:41147 req_id = 9271384f-8a35-48cb-873a-e32bdc2c56ee response_code = 200 uri = /network-bootstore-config
16462024-08-30T19:46:32.184ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 5629 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:35170 req_id = e38b8d98-6240-4416-8482-1b1c4f28295d response_code = 200 uri = /v2p
16472024-08-30T19:46:32.218ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
16482024-08-30T19:46:32.272ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 94411 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:62396 req_id = 300be535-ac74-425a-850e-249835c87c5b response_code = 204 uri = /switch-ports
16492024-08-30T19:46:32.273ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 877 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:62396 req_id = b21bfbfa-2c2c-4f33-9ec2-a2cd649ab0a8 response_code = 200 uri = /network-bootstore-config
16502024-08-30T19:46:33.117ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 6370 local_addr = [fd00:1122:3344:101::1]:48378 method = GET remote_addr = [fd00:1122:3344:101::d]:42071 req_id = 02db0f25-9f4c-423b-a406-b14e47d3ef91 response_code = 200 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be uri = /289c9d10-fa69-4638-aac7-3cfc6b4e23be
16512024-08-30T19:46:41.915ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:33921
16522024-08-30T19:46:41.915ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 523 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:33921 req_id = 5f420212-d1f6-4175-9d87-ec2ea4fe623d response_code = 200 uri = /baseboard
16532024-08-30T19:47:01.692ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:43564
16542024-08-30T19:47:01.692ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 136 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:43564 req_id = ab4b70e1-f59e-4a79-b12a-ee631561ab63 response_code = 200 uri = /vpc-routes
16552024-08-30T19:47:01.719ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 199 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:43564 req_id = 7bfeaba5-656c-4a67-84aa-d8326236313e response_code = 204 uri = /vpc-routes
16562024-08-30T19:47:01.936ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:45534
16572024-08-30T19:47:01.936ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 141 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:45534 req_id = f7f723fb-9253-4de7-9af6-622e6e8b3d80 response_code = 200 uri = /v2p
16582024-08-30T19:47:02.040ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:40213
16592024-08-30T19:47:02.040ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
16602024-08-30T19:47:02.078ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
16612024-08-30T19:47:02.136ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 96607 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:40213 req_id = e7aa843a-b7f3-4f67-a0d2-f7df4b2fcd8a response_code = 204 uri = /switch-ports
16622024-08-30T19:47:02.137ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:35982
16632024-08-30T19:47:02.137ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 50 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:35982 req_id = 6b8fba9a-c9c7-49bd-bbba-4010828c71a1 response_code = 200 uri = /vpc-routes
16642024-08-30T19:47:02.137ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 103 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:40213 req_id = 599c84af-ccfa-48d8-b742-f3b8f0be073f response_code = 200 uri = /network-bootstore-config
16652024-08-30T19:47:02.158ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:44656
16662024-08-30T19:47:02.158ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 179 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:44656 req_id = a6b6be0b-a4b8-4b19-8d4b-203d6d65a923 response_code = 200 uri = /v2p
16672024-08-30T19:47:02.167ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:45892
16682024-08-30T19:47:02.167ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 105 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:45892 req_id = faed26d4-b6fc-440d-afc4-de4ece911284 response_code = 200 uri = /v2p
16692024-08-30T19:47:02.175ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 132 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:35982 req_id = fe67e58b-41df-42ea-8e66-ba3cae4614a4 response_code = 204 uri = /vpc-routes
16702024-08-30T19:47:02.245ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:43125
16712024-08-30T19:47:02.245ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
16722024-08-30T19:47:02.283ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
16732024-08-30T19:47:02.336ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 91293 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:43125 req_id = 59b9aa75-2a2e-4382-a989-f677beae93ed response_code = 204 uri = /switch-ports
16742024-08-30T19:47:02.337ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:51157
16752024-08-30T19:47:02.338ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 721 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:51157 req_id = 4d7129b8-6438-4037-a856-dd007f20c3b2 response_code = 200 uri = /vpc-routes
16762024-08-30T19:47:02.338ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 862 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:43125 req_id = ad55d30d-b84a-4c31-a107-c4b210fe96a5 response_code = 200 uri = /network-bootstore-config
16772024-08-30T19:47:02.347ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 103 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:51157 req_id = 5d82f27e-443f-4add-aff0-3852f8675271 response_code = 204 uri = /vpc-routes
16782024-08-30T19:47:02.411ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:37710
16792024-08-30T19:47:02.411ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
16802024-08-30T19:47:02.451ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
16812024-08-30T19:47:02.509ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 98379 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:37710 req_id = 9d6c4b48-0e4e-46cc-84a2-dd367d9d5718 response_code = 204 uri = /switch-ports
16822024-08-30T19:47:02.509ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 113 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:37710 req_id = 53b34dca-a709-4668-9e1a-bab73278c6ce response_code = 200 uri = /network-bootstore-config
16832024-08-30T19:47:03.117ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 6432 local_addr = [fd00:1122:3344:101::1]:48378 method = GET remote_addr = [fd00:1122:3344:101::d]:42071 req_id = 1db10073-da5b-4a30-a496-143f642f1d56 response_code = 200 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be uri = /289c9d10-fa69-4638-aac7-3cfc6b4e23be
16842024-08-30T19:47:11.949ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:41471
16852024-08-30T19:47:11.949ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 136 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:41471 req_id = 3fd27b55-05c8-495c-ab7a-5117fcb9aacd response_code = 200 uri = /baseboard
16862024-08-30T19:47:31.720ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:36714
16872024-08-30T19:47:31.720ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 107 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:36714 req_id = ead42453-9a4e-4f02-b348-e88a1c9b5419 response_code = 200 uri = /vpc-routes
16882024-08-30T19:47:31.757ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:47307
16892024-08-30T19:47:31.757ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 180 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:47307 req_id = 85e11cbd-aa71-4090-bdd8-ada2fdca446b response_code = 200 uri = /v2p
16902024-08-30T19:47:31.767ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 116 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:36714 req_id = fbfc44e0-c9cb-4c05-978f-9cd849afa1d9 response_code = 204 uri = /vpc-routes
16912024-08-30T19:47:31.903ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:57950
16922024-08-30T19:47:31.903ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
16932024-08-30T19:47:31.948ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
16942024-08-30T19:47:32.006ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 103455 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:57950 req_id = 463c6922-aa99-42fc-aafc-fa127464b35f response_code = 204 uri = /switch-ports
16952024-08-30T19:47:32.007ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:62945
16962024-08-30T19:47:32.007ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:61514
16972024-08-30T19:47:32.007ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:46935
16982024-08-30T19:47:32.007ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 258 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:62945 req_id = 399c657e-d80f-4815-be3f-a0b08afcf4cb response_code = 200 uri = /v2p
16992024-08-30T19:47:32.007ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 159 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:46935 req_id = d22b60ed-b001-42ee-a960-ad4be3212fd8 response_code = 200 uri = /vpc-routes
17002024-08-30T19:47:32.008ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1146 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:57950 req_id = de70fc21-f0ed-4f88-98f3-18ca0c641736 response_code = 200 uri = /network-bootstore-config
17012024-08-30T19:47:32.008ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1192 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:61514 req_id = 7f088a31-3890-4233-b7bb-d1282cf15e87 response_code = 200 uri = /v2p
17022024-08-30T19:47:32.019ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 154 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:46935 req_id = 9c0af170-da23-4da1-9d96-707581945e25 response_code = 204 uri = /vpc-routes
17032024-08-30T19:47:32.055ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:49904
17042024-08-30T19:47:32.055ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
17052024-08-30T19:47:32.094ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
17062024-08-30T19:47:32.152ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 97186 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:49904 req_id = 1da2d695-a512-45f3-9d25-6986745fc020 response_code = 204 uri = /switch-ports
17072024-08-30T19:47:32.153ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:51034
17082024-08-30T19:47:32.153ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:60383
17092024-08-30T19:47:32.153ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
17102024-08-30T19:47:32.157ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3702 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:49904 req_id = 9f5f09bb-92f5-4a30-b85c-8d9b61f46f67 response_code = 200 uri = /network-bootstore-config
17112024-08-30T19:47:32.157ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3655 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:60383 req_id = f0e60c0c-2145-4b0f-81cd-ce0fb796adb6 response_code = 200 uri = /vpc-routes
17122024-08-30T19:47:32.165ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 125 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:60383 req_id = 71fb5533-8b4f-41b1-a795-6c35e7c461e2 response_code = 204 uri = /vpc-routes
17132024-08-30T19:47:32.187ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
17142024-08-30T19:47:32.240ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 87274 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:51034 req_id = a8d5dfae-0bed-415a-a3c0-6361651aa8f3 response_code = 204 uri = /switch-ports
17152024-08-30T19:47:32.241ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 866 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:51034 req_id = b7bb94ea-676b-466d-87f9-e833b7862b4f response_code = 200 uri = /network-bootstore-config
17162024-08-30T19:47:33.119ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8282 local_addr = [fd00:1122:3344:101::1]:48378 method = GET remote_addr = [fd00:1122:3344:101::d]:42071 req_id = cbdebe76-a264-4668-901f-46f21c706652 response_code = 200 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be uri = /289c9d10-fa69-4638-aac7-3cfc6b4e23be
17172024-08-30T19:47:41.983ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:56690
17182024-08-30T19:47:41.983ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 84 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:56690 req_id = 804f8539-4547-4fe4-819c-c2630f26817a response_code = 200 uri = /baseboard
17192024-08-30T19:48:01.665ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:61933
17202024-08-30T19:48:01.666ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 102 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:61933 req_id = 25c0cba1-a620-4fed-8cc8-35d6e061bd21 response_code = 200 uri = /vpc-routes
17212024-08-30T19:48:01.690ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 217 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:61933 req_id = 9f806945-ec03-4d2c-89b1-434c91c43ebe response_code = 204 uri = /vpc-routes
17222024-08-30T19:48:01.907ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:59577
17232024-08-30T19:48:01.907ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 142 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:59577 req_id = 5d2f12c5-e552-4730-87dd-78f2542bb10e response_code = 200 uri = /vpc-routes
17242024-08-30T19:48:01.930ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1461 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:59577 req_id = 8b5d7306-ae7b-43a4-811c-1b0893c5ef5a response_code = 204 uri = /vpc-routes
17252024-08-30T19:48:02.044ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:57536
17262024-08-30T19:48:02.045ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1157 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:57536 req_id = 69181c96-d6ca-4e06-95fc-73cd46ec4fba response_code = 200 uri = /v2p
17272024-08-30T19:48:02.061ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:63623
17282024-08-30T19:48:02.061ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
17292024-08-30T19:48:02.104ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
17302024-08-30T19:48:02.170ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 109185 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:63623 req_id = c6798184-7942-4f7e-9ce1-0881c346e989 response_code = 204 uri = /switch-ports
17312024-08-30T19:48:02.171ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:52111
17322024-08-30T19:48:02.171ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:37969
17332024-08-30T19:48:02.171ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
17342024-08-30T19:48:02.171ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 310 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:63623 req_id = ca74ebce-4c25-480c-a2e0-957e0f4be0cc response_code = 200 uri = /network-bootstore-config
17352024-08-30T19:48:02.176ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 4879 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:37969 req_id = df43103e-74c1-40c6-9a7a-876103f9b3cc response_code = 200 uri = /v2p
17362024-08-30T19:48:02.198ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:45132
17372024-08-30T19:48:02.198ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 110 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:45132 req_id = c36068dd-1461-4669-b333-9c778033c369 response_code = 200 uri = /vpc-routes
17382024-08-30T19:48:02.210ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 988 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:45132 req_id = 48563b7d-cf13-48b6-9bb9-3f89283da62e response_code = 204 uri = /vpc-routes
17392024-08-30T19:48:02.210ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:48769
17402024-08-30T19:48:02.211ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 148fa4d0-99ac-4f4c-b957-d26cc8730745, kind: Service { id: b5636b10-76ec-4aa4-8bd4-1f95780813df }, name: Name("external-dns-b5636b10-76ec-4aa4-8bd4-1f95780813df"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 151, 107])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e6ac84e1-d9f1-4c03-8dd9-78d5c3b887de, kind: Service { id: ff52340f-525a-4b88-bfc8-1cb0fe9ceba7 }, name: Name("external-dns-ff52340f-525a-4b88-bfc8-1cb0fe9ceba7"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 231])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 2495ea70-4560-412b-9ebd-45a2bc27810a, kind: Service { id: d2343105-862a-478c-b746-fd8bb78f9a96 }, name: Name("nexus-d2343105-862a-478c-b746-fd8bb78f9a96"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 101])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 3e58b382-51e7-46a7-8c87-6ca3a6547a20, kind: Service { id: 1d96d476-acdb-4c44-a486-29569b3bf41e }, name: Name("nexus-1d96d476-acdb-4c44-a486-29569b3bf41e"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 98])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 82aefc08-c1a4-48ce-8225-d8f4409ab816, kind: Service { id: ba4aef74-7b33-4ded-a46a-dc2534b0db9d }, name: Name("nexus-ba4aef74-7b33-4ded-a46a-dc2534b0db9d"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 158, 211])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
17412024-08-30T19:48:02.211ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
17422024-08-30T19:48:02.211ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17432024-08-30T19:48:02.211ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
17442024-08-30T19:48:02.211ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17452024-08-30T19:48:02.211ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17462024-08-30T19:48:02.211ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
17472024-08-30T19:48:02.211ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 430 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:48769 req_id = 7d11e318-1ba6-490f-8097-dc717751ca8d response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
17482024-08-30T19:48:02.214ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
17492024-08-30T19:48:02.280ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 109430 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:52111 req_id = e756614e-0818-4722-b8c0-20056a515410 response_code = 204 uri = /switch-ports
17502024-08-30T19:48:02.281ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 716 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:52111 req_id = 741c62f4-35fc-45f6-a6f4-469ff7bacaf7 response_code = 200 uri = /network-bootstore-config
17512024-08-30T19:48:02.299ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:41887
17522024-08-30T19:48:02.299ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 148fa4d0-99ac-4f4c-b957-d26cc8730745, kind: Service { id: b5636b10-76ec-4aa4-8bd4-1f95780813df }, name: Name("external-dns-b5636b10-76ec-4aa4-8bd4-1f95780813df"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 151, 107])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e6ac84e1-d9f1-4c03-8dd9-78d5c3b887de, kind: Service { id: ff52340f-525a-4b88-bfc8-1cb0fe9ceba7 }, name: Name("external-dns-ff52340f-525a-4b88-bfc8-1cb0fe9ceba7"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 231])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 2495ea70-4560-412b-9ebd-45a2bc27810a, kind: Service { id: d2343105-862a-478c-b746-fd8bb78f9a96 }, name: Name("nexus-d2343105-862a-478c-b746-fd8bb78f9a96"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 101])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 3e58b382-51e7-46a7-8c87-6ca3a6547a20, kind: Service { id: 1d96d476-acdb-4c44-a486-29569b3bf41e }, name: Name("nexus-1d96d476-acdb-4c44-a486-29569b3bf41e"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 98])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 82aefc08-c1a4-48ce-8225-d8f4409ab816, kind: Service { id: ba4aef74-7b33-4ded-a46a-dc2534b0db9d }, name: Name("nexus-ba4aef74-7b33-4ded-a46a-dc2534b0db9d"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 158, 211])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
17532024-08-30T19:48:02.299ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
17542024-08-30T19:48:02.299ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17552024-08-30T19:48:02.299ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
17562024-08-30T19:48:02.299ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17572024-08-30T19:48:02.299ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17582024-08-30T19:48:02.299ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
17592024-08-30T19:48:02.299ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 331 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:41887 req_id = 89a90be3-11fb-420b-8641-2d4a45353dec response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
17602024-08-30T19:48:02.305ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:63622
17612024-08-30T19:48:02.306ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 773 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:63622 req_id = 285866dc-17cb-426c-a8b6-89a24bc8a28a response_code = 200 uri = /v2p
17622024-08-30T19:48:02.319ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:41304
17632024-08-30T19:48:02.319ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
17642024-08-30T19:48:02.355ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
17652024-08-30T19:48:02.408ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 89134 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:41304 req_id = bb8f354a-6f92-4f7e-a5dc-7026e7287388 response_code = 204 uri = /switch-ports
17662024-08-30T19:48:02.408ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:43415
17672024-08-30T19:48:02.409ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 966 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:41304 req_id = 4db11b2c-0b7a-4e21-a088-ea9469048981 response_code = 200 uri = /network-bootstore-config
17682024-08-30T19:48:02.409ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 148fa4d0-99ac-4f4c-b957-d26cc8730745, kind: Service { id: b5636b10-76ec-4aa4-8bd4-1f95780813df }, name: Name("external-dns-b5636b10-76ec-4aa4-8bd4-1f95780813df"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 151, 107])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e6ac84e1-d9f1-4c03-8dd9-78d5c3b887de, kind: Service { id: ff52340f-525a-4b88-bfc8-1cb0fe9ceba7 }, name: Name("external-dns-ff52340f-525a-4b88-bfc8-1cb0fe9ceba7"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 231])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 2495ea70-4560-412b-9ebd-45a2bc27810a, kind: Service { id: d2343105-862a-478c-b746-fd8bb78f9a96 }, name: Name("nexus-d2343105-862a-478c-b746-fd8bb78f9a96"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 101])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 3e58b382-51e7-46a7-8c87-6ca3a6547a20, kind: Service { id: 1d96d476-acdb-4c44-a486-29569b3bf41e }, name: Name("nexus-1d96d476-acdb-4c44-a486-29569b3bf41e"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 98])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 82aefc08-c1a4-48ce-8225-d8f4409ab816, kind: Service { id: ba4aef74-7b33-4ded-a46a-dc2534b0db9d }, name: Name("nexus-ba4aef74-7b33-4ded-a46a-dc2534b0db9d"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 158, 211])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
17692024-08-30T19:48:02.409ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
17702024-08-30T19:48:02.409ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17712024-08-30T19:48:02.409ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
17722024-08-30T19:48:02.409ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17732024-08-30T19:48:02.409ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17742024-08-30T19:48:02.409ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
17752024-08-30T19:48:02.410ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1119 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:43415 req_id = b1e05d2e-18dd-4e74-a920-4f740bfafce0 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
17762024-08-30T19:48:03.118ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 6494 local_addr = [fd00:1122:3344:101::1]:48378 method = GET remote_addr = [fd00:1122:3344:101::d]:42071 req_id = 2b60668b-800e-49aa-9b5c-e18c133e23e9 response_code = 200 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be uri = /289c9d10-fa69-4638-aac7-3cfc6b4e23be
17772024-08-30T19:48:12.016ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:61593
17782024-08-30T19:48:12.016ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 78 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:61593 req_id = a90ce085-97f9-488c-b366-d0a592528a3d response_code = 200 uri = /baseboard
17792024-08-30T19:48:31.663ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:49189
17802024-08-30T19:48:31.664ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 94 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:49189 req_id = c9c7725f-6e1f-4d99-8e75-14eb44205fb1 response_code = 200 uri = /vpc-routes
17812024-08-30T19:48:31.690ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 186 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:49189 req_id = db2d56ed-1460-4a21-8977-c057eb96b726 response_code = 204 uri = /vpc-routes
17822024-08-30T19:48:31.777ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:33286
17832024-08-30T19:48:31.777ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 166 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:33286 req_id = cc466d00-e07a-4bbb-9ad8-77fd0bdaa661 response_code = 200 uri = /v2p
17842024-08-30T19:48:31.777ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:53062
17852024-08-30T19:48:31.777ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
17862024-08-30T19:48:31.832ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
17872024-08-30T19:48:31.906ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 129011 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:53062 req_id = 0f9949c3-3ab4-45ee-b451-56b81e27c24b response_code = 204 uri = /switch-ports
17882024-08-30T19:48:31.907ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:55665
17892024-08-30T19:48:31.907ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 117 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:55665 req_id = 4f4a2233-6467-4850-bb08-6241576ecd3f response_code = 200 uri = /vpc-routes
17902024-08-30T19:48:31.907ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 169 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:53062 req_id = aaebd04b-0957-40bb-8bb2-604a65c3e692 response_code = 200 uri = /network-bootstore-config
17912024-08-30T19:48:31.915ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 129 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:55665 req_id = 4e08a33b-f007-4b1d-9f47-6c4138f39485 response_code = 204 uri = /vpc-routes
17922024-08-30T19:48:31.954ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:47586
17932024-08-30T19:48:31.954ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
17942024-08-30T19:48:31.994ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
17952024-08-30T19:48:32.055ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 100832 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:47586 req_id = 61b819c0-37c8-47a6-acd9-38bddf414676 response_code = 204 uri = /switch-ports
17962024-08-30T19:48:32.055ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:60532
17972024-08-30T19:48:32.055ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:64874
17982024-08-30T19:48:32.055ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 167 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:60532 req_id = 7dcb2ce3-0a40-4e93-9ad8-3f85680fb81d response_code = 200 uri = /v2p
17992024-08-30T19:48:32.055ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 158 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:64874 req_id = e50ccbf1-ae65-4b1d-9ff0-0cafca81f347 response_code = 200 uri = /vpc-routes
18002024-08-30T19:48:32.056ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1070 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:47586 req_id = 528238b5-f728-45d8-b4b8-8f38c1b6fab2 response_code = 200 uri = /network-bootstore-config
18012024-08-30T19:48:32.063ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:41909
18022024-08-30T19:48:32.063ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 120 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:41909 req_id = df7c8f8a-b261-4d88-82fe-6d40286e0bfe response_code = 200 uri = /v2p
18032024-08-30T19:48:32.067ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 100 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:64874 req_id = 53e4baa4-1d26-4f8f-970e-c61df74789d4 response_code = 204 uri = /vpc-routes
18042024-08-30T19:48:32.152ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:40370
18052024-08-30T19:48:32.152ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
18062024-08-30T19:48:32.189ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
18072024-08-30T19:48:32.250ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 97226 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:40370 req_id = 1675494c-66a9-4fcc-b461-a87817c2de14 response_code = 204 uri = /switch-ports
18082024-08-30T19:48:32.250ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 100 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:40370 req_id = f81410c7-0217-4bd0-8ad7-cffb3abaed59 response_code = 200 uri = /network-bootstore-config
18092024-08-30T19:48:33.118ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 6431 local_addr = [fd00:1122:3344:101::1]:48378 method = GET remote_addr = [fd00:1122:3344:101::d]:42071 req_id = 5d9cd63c-a9f7-437e-bec4-a8ff17f9811c response_code = 200 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be uri = /289c9d10-fa69-4638-aac7-3cfc6b4e23be
18102024-08-30T19:48:42.052ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:56468
18112024-08-30T19:48:42.052ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 65 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:56468 req_id = 27ff5a42-92e7-4368-ae5d-ebe3ad77407c response_code = 200 uri = /baseboard
18122024-08-30T19:49:01.800ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:34702
18132024-08-30T19:49:01.800ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 83 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:34702 req_id = ef056103-34c8-403b-b4f1-f4f17c8d4134 response_code = 200 uri = /vpc-routes
18142024-08-30T19:49:01.815ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 181 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:34702 req_id = 11b931de-ce6e-442d-866f-3660d7570bca response_code = 204 uri = /vpc-routes
18152024-08-30T19:49:01.999ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:61604
18162024-08-30T19:49:01.999ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 61 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:61604 req_id = 9123a63a-09ce-4187-9c8d-93345ff72723 response_code = 200 uri = /vpc-routes
18172024-08-30T19:49:02.017ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1537 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:61604 req_id = dd579a27-1887-49e6-9f65-e224be6830d3 response_code = 204 uri = /vpc-routes
18182024-08-30T19:49:02.026ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:63741
18192024-08-30T19:49:02.026ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 129 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:63741 req_id = f3179c59-ad63-440a-9c8e-7d1bedb9649c response_code = 200 uri = /v2p
18202024-08-30T19:49:02.176ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:34850
18212024-08-30T19:49:02.176ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 136 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:34850 req_id = 569b88c7-866b-4528-b721-3758e9c4a1a0 response_code = 200 uri = /v2p
18222024-08-30T19:49:02.193ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:55343
18232024-08-30T19:49:02.193ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 42 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:55343 req_id = 36b1b76f-b25a-484b-af30-219d9bf9f242 response_code = 200 uri = /vpc-routes
18242024-08-30T19:49:02.214ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 945 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:55343 req_id = 89179e5f-5fce-4fd3-a695-76a00cd40f5d response_code = 204 uri = /vpc-routes
18252024-08-30T19:49:02.236ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:46009
18262024-08-30T19:49:02.236ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
18272024-08-30T19:49:02.276ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
18282024-08-30T19:49:02.336ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 100140 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:46009 req_id = 80e22d82-88e1-495c-9aaa-bb0aeab46bc2 response_code = 204 uri = /switch-ports
18292024-08-30T19:49:02.337ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:61786
18302024-08-30T19:49:02.337ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
18312024-08-30T19:49:02.341ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 4467 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:46009 req_id = 3e9ad82a-e65f-4242-9ce8-1a9f1fa5fde7 response_code = 200 uri = /network-bootstore-config
18322024-08-30T19:49:02.372ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
18332024-08-30T19:49:02.426ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 88880 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:61786 req_id = e89ad3f1-84a6-47ac-a564-8ee932095eb1 response_code = 204 uri = /switch-ports
18342024-08-30T19:49:02.426ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 137 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:61786 req_id = 25d00c32-81d9-4be0-a063-c23a0c4135bc response_code = 200 uri = /network-bootstore-config
18352024-08-30T19:49:02.457ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:65460
18362024-08-30T19:49:02.457ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 97 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:65460 req_id = 77ad1177-0e17-424b-a24e-99a1b199409b response_code = 200 uri = /v2p
18372024-08-30T19:49:02.494ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:39279
18382024-08-30T19:49:02.494ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
18392024-08-30T19:49:02.530ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
18402024-08-30T19:49:02.594ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 99763 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:39279 req_id = ccbf676d-f1f4-4eb0-85d3-8dda53068710 response_code = 204 uri = /switch-ports
18412024-08-30T19:49:02.594ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 99 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:39279 req_id = 6286ed22-8c96-494c-b668-fb7b7e065047 response_code = 200 uri = /network-bootstore-config
18422024-08-30T19:49:03.225ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 5973 local_addr = [fd00:1122:3344:101::1]:48378 method = GET remote_addr = [fd00:1122:3344:101::d]:42071 req_id = 2122d8ea-045c-45a0-b842-0e63b56ed6cb response_code = 200 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be uri = /289c9d10-fa69-4638-aac7-3cfc6b4e23be
18432024-08-30T19:49:12.087ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:51189
18442024-08-30T19:49:12.087ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 89 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:51189 req_id = 9a0130f8-5250-4ffc-949b-46ee93dd397c response_code = 200 uri = /baseboard
18452024-08-30T19:49:23.762ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:42997
18462024-08-30T19:49:23.762ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 702 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:42997 req_id = 3c75942f-8338-425c-aef3-b19b32c1ee30 response_code = 200 uri = /v2p
18472024-08-30T19:49:23.763ZINFOSledAgent (PortManager): Mapping virtual NIC to physical host file = illumos-utils/src/opte/port_manager.rs:778 mapping = VirtualNetworkInterfaceHost { virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 250, 253, 123])), physical_host_ip: fd00:1122:3344:101::1, vni: Vni(6417893) }
18482024-08-30T19:49:23.763ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 99 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:42997 req_id = b05bbe2f-af68-49a3-a334-b70f915c3baf response_code = 204 uri = /v2p
18492024-08-30T19:49:23.769ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:62280
18502024-08-30T19:49:23.782ZINFOSledAgent (InstanceManager): ensuring instance is registered file = sled-agent/src/instance_manager.rs:570 hardware = InstanceHardware { properties: InstanceProperties { ncpus: InstanceCpuCount(2), memory: ByteCount(1073741824), hostname: Hostname("localshark") }, nics: [NetworkInterface { id: 288eb9ab-775c-4f6c-b5c1-a9f300cff222, kind: Instance { id: 177df596-2bc8-436c-a193-ec68f297307f }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 250, 253, 123])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(6417893), primary: true, slot: 0, transit_ips: [] }], source_nat: SourceNatConfig { ip: 10.151.1.175, first_port: 0, last_port: 16383 }, ephemeral_ip: Some(10.151.1.176), floating_ips: [], firewall_rules: [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 288eb9ab-775c-4f6c-b5c1-a9f300cff222, kind: Instance { id: 177df596-2bc8-436c-a193-ec68f297307f }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 250, 253, 123])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(6417893), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 288eb9ab-775c-4f6c-b5c1-a9f300cff222, kind: Instance { id: 177df596-2bc8-436c-a193-ec68f297307f }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 250, 253, 123])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(6417893), primary: true, slot: 0, transit_ips: [] }], filter_hosts: Some([Vpc(Vni(6417893))]), filter_ports: None, filter_protocols: None, action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 288eb9ab-775c-4f6c-b5c1-a9f300cff222, kind: Instance { id: 177df596-2bc8-436c-a193-ec68f297307f }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 250, 253, 123])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(6417893), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(22), last: L4Port(22) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }], dhcp_config: DhcpConfig { dns_servers: [1.1.1.1, 9.9.9.9], host_domain: None, search_domains: [] }, disks: [DiskRequest { device: "nvme", name: "disk-3d2834543", read_only: false, slot: Slot(0), volume_construction_request: Volume { block_size: 512, id: f61f0bae-7309-409b-97f1-743b6d1487f0, read_only_parent: Some(Volume { block_size: 512, id: d687573a-13ab-47bb-9fb4-d7d5bd88758d, read_only_parent: None, sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 2, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 442b76d2-1f8c-4dd7-a018-58eba87a62a0, key: Some("Vw4mNtWoWK9GflJpgOG+5KBpMG9te9h4Z1VV4YBS69o="), key_pem: None, lossy: false, read_only: true, root_cert_pem: None, target: ["[fd00:1122:3344:101::15]:19001", "[fd00:1122:3344:101::16]:19001", "[fd00:1122:3344:101::13]:19001"] } }] }), sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 1, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: f61f0bae-7309-409b-97f1-743b6d1487f0, key: Some("JJCFBPdyXdHHw4l394/kF+K8iy5bTMoRWM8ewpNOG2k="), key_pem: None, lossy: false, read_only: false, root_cert_pem: None, target: ["[fd00:1122:3344:101::19]:19001", "[fd00:1122:3344:101::17]:19000", "[fd00:1122:3344:101::13]:19002"] } }] } }], cloud_init_bytes: Some(..) } instance_id = 177df596-2bc8-436c-a193-ec68f297307f instance_runtime = InstanceRuntimeState { propolis_id: Some(abc56fb8-ba04-4cc7-859c-561136aa9b40 (propolis)), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2024-08-30T19:49:23.303906964Z } metadata = InstanceMetadata { silo_id: aa09ea70-5935-4726-aa8b-daae29285c7a, project_id: 0ba5e404-8695-42aa-a849-7665ae57f2aa } propolis_addr = [fd00:1122:3344:101::1:0]:12400 propolis_id = abc56fb8-ba04-4cc7-859c-561136aa9b40 vmm_runtime = VmmRuntimeState { state: Starting, gen: Generation(1), time_updated: 2024-08-30T19:49:23.226374Z }
18512024-08-30T19:49:23.782ZINFOSledAgent (InstanceManager): registering new instance file = sled-agent/src/instance_manager.rs:603 instance_id = 177df596-2bc8-436c-a193-ec68f297307f propolis_id = abc56fb8-ba04-4cc7-859c-561136aa9b40
18522024-08-30T19:49:23.782ZINFOSledAgent (InstanceManager): initializing new Instance file = sled-agent/src/instance.rs:991 instance_id = 177df596-2bc8-436c-a193-ec68f297307f migration_id = None propolis_id = abc56fb8-ba04-4cc7-859c-561136aa9b40 state = InstanceInitialState { hardware: InstanceHardware { properties: InstanceProperties { ncpus: InstanceCpuCount(2), memory: ByteCount(1073741824), hostname: Hostname("localshark") }, nics: [NetworkInterface { id: 288eb9ab-775c-4f6c-b5c1-a9f300cff222, kind: Instance { id: 177df596-2bc8-436c-a193-ec68f297307f }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 250, 253, 123])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(6417893), primary: true, slot: 0, transit_ips: [] }], source_nat: SourceNatConfig { ip: 10.151.1.175, first_port: 0, last_port: 16383 }, ephemeral_ip: Some(10.151.1.176), floating_ips: [], firewall_rules: [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 288eb9ab-775c-4f6c-b5c1-a9f300cff222, kind: Instance { id: 177df596-2bc8-436c-a193-ec68f297307f }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 250, 253, 123])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(6417893), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 288eb9ab-775c-4f6c-b5c1-a9f300cff222, kind: Instance { id: 177df596-2bc8-436c-a193-ec68f297307f }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 250, 253, 123])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(6417893), primary: true, slot: 0, transit_ips: [] }], filter_hosts: Some([Vpc(Vni(6417893))]), filter_ports: None, filter_protocols: None, action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 288eb9ab-775c-4f6c-b5c1-a9f300cff222, kind: Instance { id: 177df596-2bc8-436c-a193-ec68f297307f }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 250, 253, 123])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(6417893), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(22), last: L4Port(22) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }], dhcp_config: DhcpConfig { dns_servers: [1.1.1.1, 9.9.9.9], host_domain: None, search_domains: [] }, disks: [DiskRequest { device: "nvme", name: "disk-3d2834543", read_only: false, slot: Slot(0), volume_construction_request: Volume { block_size: 512, id: f61f0bae-7309-409b-97f1-743b6d1487f0, read_only_parent: Some(Volume { block_size: 512, id: d687573a-13ab-47bb-9fb4-d7d5bd88758d, read_only_parent: None, sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 2, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 442b76d2-1f8c-4dd7-a018-58eba87a62a0, key: Some("Vw4mNtWoWK9GflJpgOG+5KBpMG9te9h4Z1VV4YBS69o="), key_pem: None, lossy: false, read_only: true, root_cert_pem: None, target: ["[fd00:1122:3344:101::15]:19001", "[fd00:1122:3344:101::16]:19001", "[fd00:1122:3344:101::13]:19001"] } }] }), sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 1, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: f61f0bae-7309-409b-97f1-743b6d1487f0, key: Some("JJCFBPdyXdHHw4l394/kF+K8iy5bTMoRWM8ewpNOG2k="), key_pem: None, lossy: false, read_only: false, root_cert_pem: None, target: ["[fd00:1122:3344:101::19]:19001", "[fd00:1122:3344:101::17]:19000", "[fd00:1122:3344:101::13]:19002"] } }] } }], cloud_init_bytes: Some(..) }, vmm_runtime: VmmRuntimeState { state: Starting, gen: Generation(1), time_updated: 2024-08-30T19:49:23.226374Z }, propolis_addr: [fd00:1122:3344:101::1:0]:12400, migration_id: None }
18532024-08-30T19:49:23.787ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 18156 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:62280 req_id = 7de89d0d-d3a2-4443-b395-7fd8f27d9419 response_code = 200 uri = /vmms/abc56fb8-ba04-4cc7-859c-561136aa9b40
18542024-08-30T19:49:23.886ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:58914
18552024-08-30T19:49:23.888ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1913 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:58914 req_id = b3338f8f-3b3a-4d9d-8233-47f9cf56b40e response_code = 200 uri = /vmms/abc56fb8-ba04-4cc7-859c-561136aa9b40/state
18562024-08-30T19:49:23.888ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte6", ip: 172.30.0.5, mac: MacAddr6([168, 64, 37, 250, 253, 123]), slot: 0, vni: Vni { inner: 6417893 }, subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.0.1 } }) }
18572024-08-30T19:49:23.889ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
18582024-08-30T19:49:23.925ZINFOSledAgent (InstanceManager): Configuring new Omicron zone: oxz_propolis-server_abc56fb8-ba04-4cc7-859c-561136aa9b40 file = illumos-utils/src/zone.rs:316 instance_id = 177df596-2bc8-436c-a193-ec68f297307f propolis_id = abc56fb8-ba04-4cc7-859c-561136aa9b40
18592024-08-30T19:49:23.950ZINFOSledAgent (InstanceManager): Installing Omicron zone: oxz_propolis-server_abc56fb8-ba04-4cc7-859c-561136aa9b40 file = illumos-utils/src/zone.rs:354 instance_id = 177df596-2bc8-436c-a193-ec68f297307f propolis_id = abc56fb8-ba04-4cc7-859c-561136aa9b40
18602024-08-30T19:49:25.474ZINFOSledAgent (InstanceManager): Profile for oxz_propolis-server_abc56fb8-ba04-4cc7-859c-561136aa9b40: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="system/illumos/propolis-server"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlInstance0'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::1:0'/> <propval type="astring" name="listen_port" value='12400'/> <propval type="astring" name="metric_addr" value='dns'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34 instance_id = 177df596-2bc8-436c-a193-ec68f297307f propolis_id = abc56fb8-ba04-4cc7-859c-561136aa9b40
18612024-08-30T19:49:25.474ZINFOSledAgent (InstanceManager): Booting oxz_propolis-server_abc56fb8-ba04-4cc7-859c-561136aa9b40 zone file = illumos-utils/src/running_zone.rs:488 instance_id = 177df596-2bc8-436c-a193-ec68f297307f propolis_id = abc56fb8-ba04-4cc7-859c-561136aa9b40 zone = oxz_propolis-server_abc56fb8-ba04-4cc7-859c-561136aa9b40
18622024-08-30T19:49:25.660ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_abc56fb8-ba04-4cc7-859c-561136aa9b40") failed: Property not found. retry in 52.610467ms file = illumos-utils/src/svc.rs:36 instance_id = 177df596-2bc8-436c-a193-ec68f297307f propolis_id = abc56fb8-ba04-4cc7-859c-561136aa9b40 zone = oxz_propolis-server_abc56fb8-ba04-4cc7-859c-561136aa9b40
18632024-08-30T19:49:25.729ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_abc56fb8-ba04-4cc7-859c-561136aa9b40") failed: Property not found. retry in 56.947429ms file = illumos-utils/src/svc.rs:36 instance_id = 177df596-2bc8-436c-a193-ec68f297307f propolis_id = abc56fb8-ba04-4cc7-859c-561136aa9b40 zone = oxz_propolis-server_abc56fb8-ba04-4cc7-859c-561136aa9b40
18642024-08-30T19:49:25.801ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_abc56fb8-ba04-4cc7-859c-561136aa9b40") failed: Property not found. retry in 231.649355ms file = illumos-utils/src/svc.rs:36 instance_id = 177df596-2bc8-436c-a193-ec68f297307f propolis_id = abc56fb8-ba04-4cc7-859c-561136aa9b40 zone = oxz_propolis-server_abc56fb8-ba04-4cc7-859c-561136aa9b40
18652024-08-30T19:49:26.050ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_abc56fb8-ba04-4cc7-859c-561136aa9b40") failed: Property not found. retry in 553.669707ms file = illumos-utils/src/svc.rs:36 instance_id = 177df596-2bc8-436c-a193-ec68f297307f propolis_id = abc56fb8-ba04-4cc7-859c-561136aa9b40 zone = oxz_propolis-server_abc56fb8-ba04-4cc7-859c-561136aa9b40
18662024-08-30T19:49:26.614ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_abc56fb8-ba04-4cc7-859c-561136aa9b40") failed: Property not found. retry in 689.137704ms file = illumos-utils/src/svc.rs:36 instance_id = 177df596-2bc8-436c-a193-ec68f297307f propolis_id = abc56fb8-ba04-4cc7-859c-561136aa9b40 zone = oxz_propolis-server_abc56fb8-ba04-4cc7-859c-561136aa9b40
18672024-08-30T19:49:27.315ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_abc56fb8-ba04-4cc7-859c-561136aa9b40") failed: Property not found. retry in 1.223097268s file = illumos-utils/src/svc.rs:36 instance_id = 177df596-2bc8-436c-a193-ec68f297307f propolis_id = abc56fb8-ba04-4cc7-859c-561136aa9b40 zone = oxz_propolis-server_abc56fb8-ba04-4cc7-859c-561136aa9b40
18682024-08-30T19:49:28.550ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_abc56fb8-ba04-4cc7-859c-561136aa9b40") failed: Property not found. retry in 1.494397612s file = illumos-utils/src/svc.rs:36 instance_id = 177df596-2bc8-436c-a193-ec68f297307f propolis_id = abc56fb8-ba04-4cc7-859c-561136aa9b40 zone = oxz_propolis-server_abc56fb8-ba04-4cc7-859c-561136aa9b40
18692024-08-30T19:49:30.056ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_abc56fb8-ba04-4cc7-859c-561136aa9b40") failed: Property not found. retry in 504.516312ms file = illumos-utils/src/svc.rs:36 instance_id = 177df596-2bc8-436c-a193-ec68f297307f propolis_id = abc56fb8-ba04-4cc7-859c-561136aa9b40 zone = oxz_propolis-server_abc56fb8-ba04-4cc7-859c-561136aa9b40
18702024-08-30T19:49:30.591ZINFOSledAgent (InstanceManager): Started propolis in zone: oxz_propolis-server_abc56fb8-ba04-4cc7-859c-561136aa9b40 file = sled-agent/src/instance.rs:1449 instance_id = 177df596-2bc8-436c-a193-ec68f297307f propolis_id = abc56fb8-ba04-4cc7-859c-561136aa9b40
18712024-08-30T19:49:30.601ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_abc56fb8-ba04-4cc7-859c-561136aa9b40") failed: Property not found. retry in 62.844434ms file = illumos-utils/src/svc.rs:36 instance_id = 177df596-2bc8-436c-a193-ec68f297307f propolis_id = abc56fb8-ba04-4cc7-859c-561136aa9b40
18722024-08-30T19:49:30.676ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_abc56fb8-ba04-4cc7-859c-561136aa9b40") failed: Property not found. retry in 116.914655ms file = illumos-utils/src/svc.rs:36 instance_id = 177df596-2bc8-436c-a193-ec68f297307f propolis_id = abc56fb8-ba04-4cc7-859c-561136aa9b40
18732024-08-30T19:49:30.802ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_abc56fb8-ba04-4cc7-859c-561136aa9b40") failed: Property not found. retry in 176.05983ms file = illumos-utils/src/svc.rs:36 instance_id = 177df596-2bc8-436c-a193-ec68f297307f propolis_id = abc56fb8-ba04-4cc7-859c-561136aa9b40
18742024-08-30T19:49:30.989ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_abc56fb8-ba04-4cc7-859c-561136aa9b40") failed: Property not found. retry in 522.952985ms file = illumos-utils/src/svc.rs:36 instance_id = 177df596-2bc8-436c-a193-ec68f297307f propolis_id = abc56fb8-ba04-4cc7-859c-561136aa9b40
18752024-08-30T19:49:31.526ZINFOSledAgent (InstanceManager): Propolis SMF service is online file = sled-agent/src/instance.rs:1458 instance_id = 177df596-2bc8-436c-a193-ec68f297307f propolis_id = abc56fb8-ba04-4cc7-859c-561136aa9b40
18762024-08-30T19:49:31.568ZINFOSledAgent (InstanceManager): Propolis HTTP server online file = sled-agent/src/instance.rs:1482 instance_id = 177df596-2bc8-436c-a193-ec68f297307f propolis_id = abc56fb8-ba04-4cc7-859c-561136aa9b40
18772024-08-30T19:49:31.677ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:43593
18782024-08-30T19:49:31.677ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 78 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:43593 req_id = 3fb59bbe-ca61-49b5-80b7-0ab06d20c0b4 response_code = 200 uri = /vpc-routes
18792024-08-30T19:49:31.705ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:57983
18802024-08-30T19:49:31.732ZINFOSledAgent (InstanceManager): result of instance_ensure call is Ok(InstanceEnsureResponse { migrate: None }) file = sled-agent/src/instance.rs:727 instance_id = 177df596-2bc8-436c-a193-ec68f297307f propolis_id = abc56fb8-ba04-4cc7-859c-561136aa9b40
18812024-08-30T19:49:31.733ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 28435 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:57983 req_id = 24d31a7d-24d7-4f7d-acc3-d535912652ca response_code = 200 uri = /vmms/abc56fb8-ba04-4cc7-859c-561136aa9b40/state
18822024-08-30T19:49:31.734ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Creating), migration_in: None, migration_out: None, time: 2024-08-30T19:49:31.734864731Z } file = sled-agent/src/instance.rs:594 instance_id = 177df596-2bc8-436c-a193-ec68f297307f propolis_id = abc56fb8-ba04-4cc7-859c-561136aa9b40
18832024-08-30T19:49:31.734ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:616 instance_id = 177df596-2bc8-436c-a193-ec68f297307f new_vmm_state = VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2024-08-30T19:49:31.734864731Z } propolis_id = abc56fb8-ba04-4cc7-859c-561136aa9b40
18842024-08-30T19:49:31.734ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:514 instance_id = 177df596-2bc8-436c-a193-ec68f297307f propolis_id = abc56fb8-ba04-4cc7-859c-561136aa9b40 state = SledVmmState { vmm_state: VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2024-08-30T19:49:31.734864731Z }, migration_in: None, migration_out: None }
18852024-08-30T19:49:31.805ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Running), migration_in: None, migration_out: None, time: 2024-08-30T19:49:31.805552721Z } file = sled-agent/src/instance.rs:594 instance_id = 177df596-2bc8-436c-a193-ec68f297307f propolis_id = abc56fb8-ba04-4cc7-859c-561136aa9b40
18862024-08-30T19:49:31.805ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:616 instance_id = 177df596-2bc8-436c-a193-ec68f297307f new_vmm_state = VmmRuntimeState { state: Running, gen: Generation(3), time_updated: 2024-08-30T19:49:31.805552721Z } propolis_id = abc56fb8-ba04-4cc7-859c-561136aa9b40
18872024-08-30T19:49:31.805ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:514 instance_id = 177df596-2bc8-436c-a193-ec68f297307f propolis_id = abc56fb8-ba04-4cc7-859c-561136aa9b40 state = SledVmmState { vmm_state: VmmRuntimeState { state: Running, gen: Generation(3), time_updated: 2024-08-30T19:49:31.805552721Z }, migration_in: None, migration_out: None }
18882024-08-30T19:49:31.812ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:43687
18892024-08-30T19:49:31.850ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:49540
18902024-08-30T19:49:31.851ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1150 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:49540 req_id = df60272b-105b-4a2f-94cf-9e4294ba9e61 response_code = 200 uri = /vpc-routes
18912024-08-30T19:49:31.904ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:64057
18922024-08-30T19:49:31.904ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 214 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:64057 req_id = 5459cd42-8998-4ef0-8c1c-f74b26874de7 response_code = 200 uri = /v2p
18932024-08-30T19:49:31.947ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 134773 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:43687 req_id = 48d287c5-2f7b-4af6-a97b-ab6b4cc2cda7 response_code = 200 uri = /vmms/abc56fb8-ba04-4cc7-859c-561136aa9b40/state
18942024-08-30T19:49:31.949ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 330 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:43593 req_id = 8196e367-425a-4f86-b522-573f0ba6fe34 response_code = 204 uri = /vpc-routes
18952024-08-30T19:49:31.975ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:64729
18962024-08-30T19:49:31.975ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
18972024-08-30T19:49:32.023ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
18982024-08-30T19:49:32.085ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 109936 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:64729 req_id = bf341ac0-cfbd-4873-b48f-5a9dbd2d3571 response_code = 204 uri = /switch-ports
18992024-08-30T19:49:32.085ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:40440
19002024-08-30T19:49:32.085ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:57519
19012024-08-30T19:49:32.085ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:65520
19022024-08-30T19:49:32.085ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
19032024-08-30T19:49:32.090ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 4081 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:65520 req_id = b71b739d-0020-47f1-a026-688230bfcf06 response_code = 200 uri = /vmms/abc56fb8-ba04-4cc7-859c-561136aa9b40/state
19042024-08-30T19:49:32.090ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 4349 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:64729 req_id = 5c1870c6-f108-4913-877d-a924a139b3fd response_code = 200 uri = /network-bootstore-config
19052024-08-30T19:49:32.090ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 4309 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:57519 req_id = b2dc418a-149c-4f97-926c-8a772c6c02c8 response_code = 200 uri = /vpc-routes
19062024-08-30T19:49:32.104ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:48011
19072024-08-30T19:49:32.108ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 137 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:57519 req_id = b8324ffe-ef00-408c-88f9-2af3e3c8a60a response_code = 204 uri = /vpc-routes
19082024-08-30T19:49:32.110ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:41562
19092024-08-30T19:49:32.110ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 107 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:41562 req_id = e149e065-2ab4-4533-bd81-b93dc0313a6c response_code = 200 uri = /v2p
19102024-08-30T19:49:32.111ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 83 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:49540 req_id = e126bd9a-bc67-4a7c-8254-bdd4e77cf47a response_code = 204 uri = /vpc-routes
19112024-08-30T19:49:32.125ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
19122024-08-30T19:49:32.179ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:59235
19132024-08-30T19:49:32.179ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 156 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:59235 req_id = f7fa8912-f464-4276-afbc-3ffa04db0fb3 response_code = 200 uri = /v2p
19142024-08-30T19:49:32.183ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
19152024-08-30T19:49:32.183ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 98075 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:40440 req_id = 0cb92839-ab81-4bc8-906d-ba94aa7970c5 response_code = 204 uri = /switch-ports
19162024-08-30T19:49:32.187ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 708 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:40440 req_id = 3880ec8d-57a5-4cfa-a463-714e32ea6445 response_code = 200 uri = /network-bootstore-config
19172024-08-30T19:49:32.216ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
19182024-08-30T19:49:32.268ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 162989 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:48011 req_id = 62504c8e-0752-4735-8d7e-49adbaaef60d response_code = 204 uri = /switch-ports
19192024-08-30T19:49:32.268ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 167 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:48011 req_id = d2d25789-129c-46ed-8830-f37fb25380e3 response_code = 200 uri = /network-bootstore-config
19202024-08-30T19:49:33.227ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 7686 local_addr = [fd00:1122:3344:101::1]:48378 method = GET remote_addr = [fd00:1122:3344:101::d]:42071 req_id = 3625a2f5-3d3c-472f-8038-a4bd1a43235a response_code = 200 sled_id = 289c9d10-fa69-4638-aac7-3cfc6b4e23be uri = /289c9d10-fa69-4638-aac7-3cfc6b4e23be
19212024-08-30T19:49:40.882ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:52659
19222024-08-30T19:49:40.884ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1194 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:52659 req_id = 18c1119d-6c2b-43bb-b47f-a8f12b95e23e response_code = 200 uri = /vmms/abc56fb8-ba04-4cc7-859c-561136aa9b40/state
19232024-08-30T19:49:40.884ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Stopping), migration_in: None, migration_out: None, time: 2024-08-30T19:49:40.884202502Z } file = sled-agent/src/instance.rs:594 instance_id = 177df596-2bc8-436c-a193-ec68f297307f propolis_id = abc56fb8-ba04-4cc7-859c-561136aa9b40
19242024-08-30T19:49:40.884ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:616 instance_id = 177df596-2bc8-436c-a193-ec68f297307f new_vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(5), time_updated: 2024-08-30T19:49:40.884202502Z } propolis_id = abc56fb8-ba04-4cc7-859c-561136aa9b40
19252024-08-30T19:49:40.884ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:514 instance_id = 177df596-2bc8-436c-a193-ec68f297307f propolis_id = abc56fb8-ba04-4cc7-859c-561136aa9b40 state = SledVmmState { vmm_state: VmmRuntimeState { state: Stopping, gen: Generation(5), time_updated: 2024-08-30T19:49:40.884202502Z }, migration_in: None, migration_out: None }
19262024-08-30T19:49:40.974ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Destroyed), migration_in: None, migration_out: None, time: 2024-08-30T19:49:40.974855983Z } file = sled-agent/src/instance.rs:594 instance_id = 177df596-2bc8-436c-a193-ec68f297307f propolis_id = abc56fb8-ba04-4cc7-859c-561136aa9b40
19272024-08-30T19:49:40.974ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:616 instance_id = 177df596-2bc8-436c-a193-ec68f297307f new_vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(6), time_updated: 2024-08-30T19:49:40.974855983Z } propolis_id = abc56fb8-ba04-4cc7-859c-561136aa9b40
19282024-08-30T19:49:40.974ZINFOSledAgent (InstanceManager): terminating VMM that has exited file = sled-agent/src/instance.rs:630 instance_id = 177df596-2bc8-436c-a193-ec68f297307f propolis_id = abc56fb8-ba04-4cc7-859c-561136aa9b40
19292024-08-30T19:49:40.974ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
19302024-08-30T19:49:40.976ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
19312024-08-30T19:49:40.977ZINFOSledAgent (ZoneBundler): creating zone bundle context = ZoneBundleContext { storage_dirs: ["/pool/int/17afe9d4-47cb-4799-8d3d-ad69badedc16/debug/bundle/zone", "/pool/int/fc9e5708-cd50-4950-8336-db011df04e25/debug/bundle/zone"], cause: TerminatedInstance, extra_log_dirs: ["/pool/ext/83c1a1b5-5f0a-46f8-a0fa-9e2ddbe675b9/crypt/debug", "/pool/ext/1693cbb3-d027-4679-ba05-f26fe21ef04a/crypt/debug", "/pool/ext/161cdb67-d423-452c-be58-7bca2891066e/crypt/debug", "/pool/ext/3fd4f884-7e56-4c58-871f-b010d32df20b/crypt/debug", "/pool/ext/6113cfc6-d07f-470a-963b-4465a62b65f1/crypt/debug", "/pool/ext/9a6b9e1b-b715-4f19-b00f-4a1c2b109fdc/crypt/debug", "/pool/ext/50a6c5cd-19db-4cbc-bda0-e365b45637b0/crypt/debug", "/pool/ext/a2ff29f8-f12c-4d2f-add0-1d5faf6635ee/crypt/debug", "/pool/ext/d69ccb3d-8bd7-402d-b7bd-712c30af96dc/crypt/debug"] } file = sled-agent/src/zone_bundle.rs:339 zone_name = oxz_propolis-server_abc56fb8-ba04-4cc7-859c-561136aa9b40
19322024-08-30T19:49:41.595ZINFOSledAgent (ZoneBundler): finished zone bundle file = sled-agent/src/zone_bundle.rs:1137 metadata = ZoneBundleMetadata { id: ZoneBundleId { zone_name: "oxz_propolis-server_abc56fb8-ba04-4cc7-859c-561136aa9b40", bundle_id: 6758943d-2ca7-4505-bf11-cd01b5393f4f }, time_created: 2024-08-30T19:49:40.977522140Z, version: 0, cause: TerminatedInstance }
19332024-08-30T19:49:41.596ZWARNSledAgent (InstanceManager): Halting and removing zone: oxz_propolis-server_abc56fb8-ba04-4cc7-859c-561136aa9b40 file = sled-agent/src/instance.rs:826 instance_id = 177df596-2bc8-436c-a193-ec68f297307f propolis_id = abc56fb8-ba04-4cc7-859c-561136aa9b40
19342024-08-30T19:49:42.119ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:42839
19352024-08-30T19:49:42.119ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 119 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:42839 req_id = a39c7aad-9f60-40e9-8a01-0f613b12d062 response_code = 200 uri = /baseboard
19362024-08-30T19:49:43.349ZINFOSledAgent (InstanceManager): halt_and_remove_logged: Previous zone state: Running file = illumos-utils/src/zone.rs:270 instance_id = 177df596-2bc8-436c-a193-ec68f297307f propolis_id = abc56fb8-ba04-4cc7-859c-561136aa9b40
19372024-08-30T19:49:43.365ZINFOSledAgent (PortManager): Removed route set for subnet file = illumos-utils/src/opte/port_manager.rs:904 id = RouterId { vni: Vni(6417893), kind: System }
19382024-08-30T19:49:43.365ZINFOSledAgent (PortManager): Removed route set for subnet file = illumos-utils/src/opte/port_manager.rs:904 id = RouterId { vni: Vni(6417893), kind: Custom(V4(Ipv4Net { addr: 172.30.0.0, width: 22 })) }
19392024-08-30T19:49:43.366ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:514 instance_id = 177df596-2bc8-436c-a193-ec68f297307f propolis_id = abc56fb8-ba04-4cc7-859c-561136aa9b40 state = SledVmmState { vmm_state: VmmRuntimeState { state: Destroyed, gen: Generation(7), time_updated: 2024-08-30T19:49:43.366545999Z }, migration_in: None, migration_out: None }
19402024-08-30T19:49:43.397ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:514 instance_id = 177df596-2bc8-436c-a193-ec68f297307f propolis_id = abc56fb8-ba04-4cc7-859c-561136aa9b40 state = SledVmmState { vmm_state: VmmRuntimeState { state: Destroyed, gen: Generation(7), time_updated: 2024-08-30T19:49:43.366545999Z }, migration_in: None, migration_out: None }
19412024-08-30T19:49:43.402ZINFOSledAgent (InstanceManager): State monitoring task complete file = sled-agent/src/instance.rs:763 instance_id = 177df596-2bc8-436c-a193-ec68f297307f propolis_id = abc56fb8-ba04-4cc7-859c-561136aa9b40
19422024-08-30T19:49:43.864ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:37186
19432024-08-30T19:49:43.865ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 63 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:37186 req_id = 8dfb5d0b-c9a3-4ae2-9a90-26e6bc922c0b response_code = 200 uri = /vpc-routes
19442024-08-30T19:49:43.873ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 103 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:37186 req_id = e27cfea6-5ff6-4223-b759-6664740d6939 response_code = 204 uri = /vpc-routes
19452024-08-30T19:49:43.969ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:63497
19462024-08-30T19:49:43.969ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 177 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:63497 req_id = 7d151324-0031-45d8-9abe-b85358195e70 response_code = 200 uri = /v2p
19472024-08-30T19:49:44.484ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:40682
19482024-08-30T19:49:44.484ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 212 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:40682 req_id = bce13c91-2af7-4be6-9e30-a504878e81ca response_code = 200 uri = /v2p
19492024-08-30T19:49:45.585ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:64641
19502024-08-30T19:49:45.585ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 89 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:64641 req_id = 406cd2c8-4c23-431b-94e6-71a53dde6d6b response_code = 200 uri = /vpc-routes
19512024-08-30T19:49:45.594ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 117 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:64641 req_id = 8677465d-3738-42a5-886d-19ae3dac9435 response_code = 204 uri = /vpc-routes