Max level shown:
1 [ Aug 28 00:51:28 Enabled. ]
2 [ Aug 28 00:51:28 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/sled-agent/sled-agent run /opt/oxide/sled-agent/pkg/config.toml &"). ]
3 [ Aug 28 00:51:28 Method "start" exited with status 0. ]
4 [ Aug 28 00:51:28 Rereading configuration. ]
5 [ Aug 28 00:51:28 No 'refresh' method defined. Treating as :true. ]
6 note: configured to log to "/dev/stdout"
72024-08-28T00:51:28.768ZINFOSledAgent: Ensuring contract reaper thread file = illumos-utils/src/running_zone.rs:113
82024-08-28T00:51:28.768ZINFOSledAgent (BootstrapAgentStartup): finding links ["net0", "net1"] file = sled-agent/src/bootstrap/pre_server.rs:255
92024-08-28T00:51:28.819ZINFOSledAgent (BootstrapAgentStartup): Starting mg-ddm service addr-objs = [AddrObject { interface: "net0", name: "ll" }, AddrObject { interface: "net1", name: "ll" }] file = sled-agent/src/bootstrap/pre_server.rs:262
102024-08-28T00:51:28.819ZINFOSledAgent (BootstrapAgentStartup): Importing mg-ddm service file = sled-agent/src/bootstrap/maghemite.rs:42 path = /opt/oxide/mg-ddm/pkg/ddm/manifest.xml
112024-08-28T00:51:28.934ZINFOSledAgent (BootstrapAgentStartup): Setting mg-ddm interfaces file = sled-agent/src/bootstrap/maghemite.rs:50 interfaces = ("net0/ll" "net1/ll")
122024-08-28T00:51:28.943ZINFOSledAgent (BootstrapAgentStartup): Enabling mg-ddm service file = sled-agent/src/bootstrap/maghemite.rs:56
132024-08-28T00:51:28.951ZINFOSledAgent (BootstrapAgentStartup): Ensuring zfs key directory exists file = sled-agent/src/bootstrap/pre_server.rs:272 path = /var/run/oxide/
142024-08-28T00:51:29.551ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fdb0:18c0:4d0d:9fb2::, width: 64 }
152024-08-28T00:51:29.692ZINFOSledAgent: Starting KeyManager file = sled-agent/src/long_running_tasks.rs:126
162024-08-28T00:51:29.692ZINFOSledAgent: Starting StorageManager file = sled-agent/src/long_running_tasks.rs:138
172024-08-28T00:51:29.692ZINFOSledAgent: Starting StorageMonitor file = sled-agent/src/long_running_tasks.rs:151
182024-08-28T00:51:29.692ZINFOSledAgent: Starting HardwareManager file = sled-agent/src/long_running_tasks.rs:173 nongimlet_observed_disks = [] sled_mode = Scrimlet { asic: SoftNpuZone }
192024-08-28T00:51:29.692ZINFOSledAgent (DumpSetup-worker): DumpSetup poll loop started. file = sled-agent/src/dump_setup.rs:520
202024-08-28T00:51:29.692ZINFOSledAgent (HardwareManager): Creating HardwareManager file = sled-hardware/src/illumos/mod.rs:716
212024-08-28T00:51:29.695ZINFOSledAgent (HardwareManager): Generated i86pc baseboard Pc { identifier: "buskin", model: "i86pc" } file = sled-hardware/src/illumos/mod.rs:623
222024-08-28T00:51:29.695ZWARNSledAgent (HardwareManager): Device is not a Gimlet (i86pc), proceeding with null hardware view file = sled-hardware/src/illumos/mod.rs:765
232024-08-28T00:51:29.695ZINFOSledAgent: Starting HardwareMonitor file = sled-agent/src/long_running_tasks.rs:187
242024-08-28T00:51:29.695ZINFOSledAgent (HardwareManager): Monitoring for hardware updates file = sled-hardware/src/illumos/mod.rs:824
252024-08-28T00:51:29.695ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/m2_0.vdev
262024-08-28T00:51:29.695ZINFOSledAgent (HardwareMonitor): Checking current full hardware snapshot disks = {} file = sled-agent/src/hardware_monitor.rs:271 underlay_network_info = None
272024-08-28T00:51:29.695ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:406
282024-08-28T00:51:29.695ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
292024-08-28T00:51:29.695ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } path = /scratch/m2_0.vdev
302024-08-28T00:51:29.707ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/m2_0.vdev file = sled-hardware/src/disk.rs:360
312024-08-28T00:51:29.707ZINFOSledAgent (StorageResources): Formatting zpool with generated ID file = sled-hardware/src/disk.rs:371 id = 1d8f47b2-ceb5-4768-b5e0-de73f9c6288a (zpool)
322024-08-28T00:51:29.803ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:307 zpool = ZpoolName { id: 1d8f47b2-ceb5-4768-b5e0-de73f9c6288a (zpool), kind: Internal }
332024-08-28T00:51:30.008ZINFOSledAgent (StorageManager): Received DetectedRawDisksChanged { raw_disks: {}, tx: ... } file = sled-storage/src/manager.rs:406
342024-08-28T00:51:30.008ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/m2_1.vdev
352024-08-28T00:51:30.008ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
362024-08-28T00:51:30.008ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
372024-08-28T00:51:30.008ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:406
382024-08-28T00:51:30.008ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
392024-08-28T00:51:30.008ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" } path = /scratch/m2_1.vdev
402024-08-28T00:51:30.017ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/m2_1.vdev file = sled-hardware/src/disk.rs:360
412024-08-28T00:51:30.017ZINFOSledAgent (StorageResources): Formatting zpool with generated ID file = sled-hardware/src/disk.rs:371 id = 2701509e-5560-4f63-bb12-b820fcc5e4cb (zpool)
422024-08-28T00:51:30.109ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:307 zpool = ZpoolName { id: 2701509e-5560-4f63-bb12-b820fcc5e4cb (zpool), kind: Internal }
432024-08-28T00:51:30.308ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_0.vdev
442024-08-28T00:51:30.308ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 1d8f47b2-ceb5-4768-b5e0-de73f9c6288a (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 2701509e-5560-4f63-bb12-b820fcc5e4cb (zpool), kind: Internal } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
452024-08-28T00:51:30.308ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:406
462024-08-28T00:51:30.308ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
472024-08-28T00:51:30.308ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
482024-08-28T00:51:30.308ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
492024-08-28T00:51:30.308ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_1.vdev
502024-08-28T00:51:30.308ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 1d8f47b2-ceb5-4768-b5e0-de73f9c6288a (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 2701509e-5560-4f63-bb12-b820fcc5e4cb (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
512024-08-28T00:51:30.308ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
522024-08-28T00:51:30.308ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
532024-08-28T00:51:30.308ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:406
542024-08-28T00:51:30.308ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
552024-08-28T00:51:30.308ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_2.vdev
562024-08-28T00:51:30.308ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 1d8f47b2-ceb5-4768-b5e0-de73f9c6288a (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 2701509e-5560-4f63-bb12-b820fcc5e4cb (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
572024-08-28T00:51:30.308ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:406
582024-08-28T00:51:30.308ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
592024-08-28T00:51:30.308ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
602024-08-28T00:51:30.308ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
612024-08-28T00:51:30.308ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_3.vdev
622024-08-28T00:51:30.308ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:406
632024-08-28T00:51:30.308ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
642024-08-28T00:51:30.308ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 1d8f47b2-ceb5-4768-b5e0-de73f9c6288a (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 2701509e-5560-4f63-bb12-b820fcc5e4cb (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
652024-08-28T00:51:30.308ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
662024-08-28T00:51:30.308ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_4.vdev
672024-08-28T00:51:30.308ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
682024-08-28T00:51:30.308ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:406
692024-08-28T00:51:30.308ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
702024-08-28T00:51:30.308ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 1d8f47b2-ceb5-4768-b5e0-de73f9c6288a (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 2701509e-5560-4f63-bb12-b820fcc5e4cb (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
712024-08-28T00:51:30.308ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
722024-08-28T00:51:30.308ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
732024-08-28T00:51:30.308ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_5.vdev
742024-08-28T00:51:30.308ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 1d8f47b2-ceb5-4768-b5e0-de73f9c6288a (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 2701509e-5560-4f63-bb12-b820fcc5e4cb (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
752024-08-28T00:51:30.308ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
762024-08-28T00:51:30.308ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
772024-08-28T00:51:30.308ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:406
782024-08-28T00:51:30.308ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
792024-08-28T00:51:30.308ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_6.vdev
802024-08-28T00:51:30.308ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 1d8f47b2-ceb5-4768-b5e0-de73f9c6288a (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 2701509e-5560-4f63-bb12-b820fcc5e4cb (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
812024-08-28T00:51:30.308ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
822024-08-28T00:51:30.308ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
832024-08-28T00:51:30.308ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:406
842024-08-28T00:51:30.308ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
852024-08-28T00:51:30.308ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_7.vdev
862024-08-28T00:51:30.308ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 1d8f47b2-ceb5-4768-b5e0-de73f9c6288a (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 2701509e-5560-4f63-bb12-b820fcc5e4cb (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
872024-08-28T00:51:30.308ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
882024-08-28T00:51:30.308ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
892024-08-28T00:51:30.308ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:406
902024-08-28T00:51:30.308ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
912024-08-28T00:51:30.308ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_8.vdev
922024-08-28T00:51:30.308ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:406
932024-08-28T00:51:30.308ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
942024-08-28T00:51:30.308ZINFOSledAgent: Waiting for boot disk file = sled-agent/src/long_running_tasks.rs:97
952024-08-28T00:51:30.308ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 1d8f47b2-ceb5-4768-b5e0-de73f9c6288a (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 2701509e-5560-4f63-bb12-b820fcc5e4cb (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
962024-08-28T00:51:30.308ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
972024-08-28T00:51:30.308ZINFOSledAgent: Found boot disk DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } file = sled-agent/src/long_running_tasks.rs:99
982024-08-28T00:51:30.308ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
992024-08-28T00:51:30.308ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
1002024-08-28T00:51:30.308ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 1d8f47b2-ceb5-4768-b5e0-de73f9c6288a (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 2701509e-5560-4f63-bb12-b820fcc5e4cb (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
1012024-08-28T00:51:30.308ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
1022024-08-28T00:51:30.308ZINFOSledAgent: Starting Bootstore file = sled-agent/src/long_running_tasks.rs:211
1032024-08-28T00:51:30.308ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
1042024-08-28T00:51:30.308ZINFOSledAgent (bootstore): No ledger in /pool/int/1d8f47b2-ceb5-4768-b5e0-de73f9c6288a/cluster/bootstore-fsm-state.json file = common/src/ledger.rs:188 peer_id = pc-buskin-i86pc
1052024-08-28T00:51:30.308ZINFOSledAgent (bootstore): No ledger in /pool/int/2701509e-5560-4f63-bb12-b820fcc5e4cb/cluster/bootstore-fsm-state.json file = common/src/ledger.rs:188 peer_id = pc-buskin-i86pc
1062024-08-28T00:51:30.308ZINFOSledAgent (bootstore): No ledger found. Loading Fsm::State as Uninitialized file = bootstore/src/schemes/v0/storage.rs:83 peer_id = pc-buskin-i86pc
1072024-08-28T00:51:30.308ZINFOSledAgent (bootstore): No ledger in /pool/int/1d8f47b2-ceb5-4768-b5e0-de73f9c6288a/cluster/bootstore-network-config.json file = common/src/ledger.rs:188 peer_id = pc-buskin-i86pc
1082024-08-28T00:51:30.308ZINFOSledAgent (bootstore): No ledger in /pool/int/2701509e-5560-4f63-bb12-b820fcc5e4cb/cluster/bootstore-network-config.json file = common/src/ledger.rs:188 peer_id = pc-buskin-i86pc
1092024-08-28T00:51:30.308ZINFOSledAgent (bootstore): No ledger found for network config file = bootstore/src/schemes/v0/storage.rs:146 peer_id = pc-buskin-i86pc
1102024-08-28T00:51:30.308ZINFOSledAgent: Starting Bootstore DDMD poller file = sled-agent/src/long_running_tasks.rs:216
1112024-08-28T00:51:30.308ZINFOSledAgent: Starting ZoneBundler related tasks file = sled-agent/src/long_running_tasks.rs:231
1122024-08-28T00:51:30.320ZINFOSledAgent (ServiceManager): Creating ServiceManager file = sled-agent/src/services.rs:731
1132024-08-28T00:51:30.320ZINFOSledAgent (auto-cleanup-task): top of bundle cleanup loop file = sled-agent/src/zone_bundle.rs:186 next_cleanup = Instant { tv_sec: 1353, tv_nsec: 939309625 } time_to_next_cleanup = 599.999922647s
1142024-08-28T00:51:30.320ZINFOSledAgent (ServiceManager): Ensuring scrimlet services (enabling services) file = sled-agent/src/services.rs:3861
1152024-08-28T00:51:30.320ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
1162024-08-28T00:51:30.320ZINFOSledAgent (BootstrapAgentStartup): No ledger in /pool/int/1d8f47b2-ceb5-4768-b5e0-de73f9c6288a/config/sled-agent-request.json file = common/src/ledger.rs:188
1172024-08-28T00:51:30.323ZINFOSledAgent (BootstrapAgentStartup): No ledger in /pool/int/2701509e-5560-4f63-bb12-b820fcc5e4cb/config/sled-agent-request.json file = common/src/ledger.rs:188
1182024-08-28T00:51:30.323ZINFOSledAgent (dropshot (BootstrapAgent)): listening file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:205 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80
1192024-08-28T00:51:30.323ZINFOSledAgent (SledAgentSprocketsServer): Started listening file = sled-agent/src/bootstrap/sprockets_server.rs:44 local_addr = [fdb0:18c0:4d0d:9fb2::1]:12346
1202024-08-28T00:51:30.323ZINFOSledAgent (RSS): Injecting RSS configuration: RackInitializeRequest { trust_quorum_peers: None, bootstrap_discovery: OnlyOurs, ntp_servers: [ "0.pool.ntp.org", ], dns_servers: [ 1.1.1.1, 9.9.9.9, ], internal_services_ip_pool_ranges: [ V4( Ipv4Range { first: 10.151.1.164, last: 10.151.1.173, }, ), ], external_dns_ips: [ 10.151.1.164, 10.151.1.165, ], external_dns_zone_name: "oxide.test", external_certificates: "<redacted>", recovery_silo: RecoverySiloConfig { silo_name: Name( "recovery", ), user_name: UserId( "recovery", ), user_password_hash: NewPasswordHash( PasswordHashString { string: "$argon2id$v=19$m=98304,t=13,p=1$RUlWc0ZxaHo0WFdrN0N6ZQ$S8p52j85GPvMhR/ek3GL0el/oProgTwWpHJZ8lsQQoY", encoding: B64, }, ), }, rack_network_config: RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, infra_ip_first: 10.151.1.174, infra_ip_last: 10.151.1.174, ports: [ PortConfigV2 { routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), nexthop: 10.151.1.100, vlan_id: None, local_pref: None, }, ], addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.1.174, width: 24, }, ), vlan_id: None, }, ], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: None, bgp_peers: [], autoneg: false, lldp: None, }, ], bgp: [], bfd: [], }, allowed_source_ips: Any, } file = sled-agent/src/rack_setup/service.rs:1053
1212024-08-28T00:51:30.323ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = internal-dns/src/resolver.rs:73
1222024-08-28T00:51:30.323ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
1232024-08-28T00:51:30.323ZINFOSledAgent (RSS): No ledger in /pool/int/1d8f47b2-ceb5-4768-b5e0-de73f9c6288a/config/rss-plan-completed.marker file = common/src/ledger.rs:188
1242024-08-28T00:51:30.323ZINFOSledAgent (RSS): No ledger in /pool/int/2701509e-5560-4f63-bb12-b820fcc5e4cb/config/rss-plan-completed.marker file = common/src/ledger.rs:188
1252024-08-28T00:51:30.323ZINFOSledAgent (RSS): RSS configuration has not been fully applied yet file = sled-agent/src/rack_setup/service.rs:1113
1262024-08-28T00:51:30.323ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
1272024-08-28T00:51:30.323ZINFOSledAgent (RSS): No ledger in /pool/int/1d8f47b2-ceb5-4768-b5e0-de73f9c6288a/config/rss-sled-plan.json file = common/src/ledger.rs:188
1282024-08-28T00:51:30.323ZINFOSledAgent (RSS): No ledger in /pool/int/2701509e-5560-4f63-bb12-b820fcc5e4cb/config/rss-sled-plan.json file = common/src/ledger.rs:188
1292024-08-28T00:51:30.323ZINFOSledAgent (RSS): Creating new allocation plan file = sled-agent/src/rack_setup/service.rs:1154
1302024-08-28T00:51:30.323ZINFOSledAgent (RSS): Serializing plan file = sled-agent/src/rack_setup/plan/sled.rs:155
1312024-08-28T00:51:30.323ZINFOSledAgent (RSS): Creating plan for the sled at fdb0:18c0:4d0d:9fb2::1 file = sled-agent/src/rack_setup/plan/sled.rs:128
1322024-08-28T00:51:30.323ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
1332024-08-28T00:51:30.323ZINFOSledAgent (RSS): Writing ledger to /pool/int/1d8f47b2-ceb5-4768-b5e0-de73f9c6288a/config/.rss-sled-plan.json.tmp file = common/src/ledger.rs:199
1342024-08-28T00:51:30.324ZINFOSledAgent (RSS): Writing ledger to /pool/int/2701509e-5560-4f63-bb12-b820fcc5e4cb/config/.rss-sled-plan.json.tmp file = common/src/ledger.rs:199
1352024-08-28T00:51:30.324ZINFOSledAgent (RSS): Sled plan written to storage: Plan { rack_id: 75a88ee6-61ab-40f4-a8f1-caf11e1c9852, sleds: { [fdb0:18c0:4d0d:9fb2::1]:12346: StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: 45733be1-aef6-43c8-be71-719893a1feda, rack_id: 75a88ee6-61ab-40f4-a8f1-caf11e1c9852, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64, }, }, }, }, }, config: RackInitializeRequest { trust_quorum_peers: None, bootstrap_discovery: OnlyOurs, ntp_servers: [ "0.pool.ntp.org", ], dns_servers: [ 1.1.1.1, 9.9.9.9, ], internal_services_ip_pool_ranges: [ V4( Ipv4Range { first: 10.151.1.164, last: 10.151.1.173, }, ), ], external_dns_ips: [ 10.151.1.164, 10.151.1.165, ], external_dns_zone_name: "oxide.test", external_certificates: "<redacted>", recovery_silo: RecoverySiloConfig { silo_name: Name( "recovery", ), user_name: UserId( "recovery", ), user_password_hash: NewPasswordHash( PasswordHashString { string: "$argon2id$v=19$m=98304,t=13,p=1$RUlWc0ZxaHo0WFdrN0N6ZQ$S8p52j85GPvMhR/ek3GL0el/oProgTwWpHJZ8lsQQoY", encoding: B64, }, ), }, rack_network_config: RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, infra_ip_first: 10.151.1.174, infra_ip_last: 10.151.1.174, ports: [ PortConfigV2 { routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), nexthop: 10.151.1.100, vlan_id: None, local_pref: None, }, ], addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.1.174, width: 24, }, ), vlan_id: None, }, ], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: None, bgp_peers: [], autoneg: false, lldp: None, }, ], bgp: [], bfd: [], }, allowed_source_ips: Any, }, } file = sled-agent/src/rack_setup/plan/sled.rs:175
1362024-08-28T00:51:30.324ZINFOSledAgent (RSS): Writing Rack Network Configuration to bootstore file = sled-agent/src/rack_setup/service.rs:1187
1372024-08-28T00:51:30.324ZINFOSledAgent (bootstore): Attempting to update network config with generation: 1, current_generation: 0 file = bootstore/src/schemes/v0/peer.rs:564 peer_id = pc-buskin-i86pc
1382024-08-28T00:51:30.324ZINFOSledAgent (bootstore): Writing ledger to /pool/int/1d8f47b2-ceb5-4768-b5e0-de73f9c6288a/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-buskin-i86pc
1392024-08-28T00:51:30.324ZINFOSledAgent (bootstore): Writing ledger to /pool/int/2701509e-5560-4f63-bb12-b820fcc5e4cb/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-buskin-i86pc
1402024-08-28T00:51:30.324ZINFOSledAgent (bootstore): Broadcasting network config with generation 1 file = bootstore/src/schemes/v0/peer.rs:648 peer_id = pc-buskin-i86pc
1412024-08-28T00:51:30.325ZINFOSledAgent (BootstrapAgentRssHandler): Received initialization request from RSS file = sled-agent/src/bootstrap/rss_handle.rs:217 request = StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: 45733be1-aef6-43c8-be71-719893a1feda, rack_id: 75a88ee6-61ab-40f4-a8f1-caf11e1c9852, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64 } } } } target_sled = [fdb0:18c0:4d0d:9fb2::1]:12346
1422024-08-28T00:51:30.325ZINFOSledAgent (SledAgentSprocketsServer): Accepted connection file = sled-agent/src/bootstrap/sprockets_server.rs:66 remote_addr = [fdb0:18c0:4d0d:9fb2::1]:42213
1432024-08-28T00:51:30.325ZINFOSledAgent (SledAgentMain): Loading Sled Agent: StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: 45733be1-aef6-43c8-be71-719893a1feda, rack_id: 75a88ee6-61ab-40f4-a8f1-caf11e1c9852, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64 } } } } file = sled-agent/src/bootstrap/server.rs:359
1442024-08-28T00:51:30.325ZINFOSledAgent (SledAgentMain): KeyManager: using hardcoded secret retriever file = sled-agent/src/bootstrap/server.rs:374
1452024-08-28T00:51:30.325ZINFOSledAgent: setting up sled agent server file = sled-agent/src/server.rs:44
1462024-08-28T00:51:30.325ZINFOSledAgent (StorageManager): Received KeyManagerReady file = sled-storage/src/manager.rs:406
1472024-08-28T00:51:30.325ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = internal-dns/src/resolver.rs:73
1482024-08-28T00:51:30.325ZINFOSledAgent (StorageManager): Loading storage config file = sled-storage/src/manager.rs:574
1492024-08-28T00:51:30.325ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:101::, width: 64 }
1502024-08-28T00:51:30.325ZINFOSledAgent (StorageManager): No ledger in /pool/int/1d8f47b2-ceb5-4768-b5e0-de73f9c6288a/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = load_ledger
1512024-08-28T00:51:30.325ZINFOSledAgent (StorageManager): No ledger in /pool/int/2701509e-5560-4f63-bb12-b820fcc5e4cb/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = load_ledger
1522024-08-28T00:51:30.325ZINFOSledAgent (StorageManager): No ledger of physical disks exists file = sled-storage/src/manager.rs:563
1532024-08-28T00:51:30.325ZINFOSledAgent (StorageManager): KeyManager ready, but no ledger detected file = sled-storage/src/manager.rs:591
1542024-08-28T00:51:30.325ZINFOSledAgent (StorageManager): automatically managing disks file = sled-storage/src/manager.rs:395
1552024-08-28T00:51:30.325ZINFOSledAgent (StorageResources): Synchronizing disk managment file = sled-storage/src/resources.rs:302
1562024-08-28T00:51:30.325ZINFOSledAgent (StorageManager): Successfully synchronized disks without error file = sled-storage/src/manager.rs:481 result = DisksManagementResult { status: [] }
1572024-08-28T00:51:30.327ZINFOSledAgent (ServiceManager): Enabling switch zone (new) file = sled-agent/src/services.rs:4158
1582024-08-28T00:51:30.327ZINFOSledAgent (ServiceManager): Starting switch zone file = sled-agent/src/services.rs:4540
1592024-08-28T00:51:30.357ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
1602024-08-28T00:51:30.377ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_switch file = illumos-utils/src/zone.rs:316
1612024-08-28T00:51:30.395ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_switch file = illumos-utils/src/zone.rs:354
1622024-08-28T00:51:30.405ZINFOSledAgent: SledAgent::new(..) starting file = sled-agent/src/sled_agent.rs:385 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
1632024-08-28T00:51:30.405ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
1642024-08-28T00:51:30.405ZINFOSledAgent: Requested swap device of size 64 GiB file = sled-agent/src/sled_agent.rs:397 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
1652024-08-28T00:51:30.405ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1662024-08-28T00:51:30.408ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 51.058803ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "00cd980d-d5c8-499a-9542-218585f6a4eb", "content-length": "124", "date": "Wed, 28 Aug 2024 00:51:29 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "00cd980d-d5c8-499a-9542-218585f6a4eb" } file = clients/ddm-admin-client/src/lib.rs:177
1672024-08-28T00:51:30.419ZINFOSledAgent: attempting to create encrypted zvol: name="oxi_1d8f47b2-ceb5-4768-b5e0-de73f9c6288a/swap", size_gb=64 file = sled-agent/src/swap_device.rs:153
1682024-08-28T00:51:30.449ZINFOSledAgent: successfully created encrypted zvol: name="oxi_1d8f47b2-ceb5-4768-b5e0-de73f9c6288a/swap", size_gb=64 file = sled-agent/src/swap_device.rs:249
1692024-08-28T00:51:30.449ZINFOSledAgent: adding swap device: swapname="/dev/zvol/dsk/oxi_1d8f47b2-ceb5-4768-b5e0-de73f9c6288a/swap" file = sled-agent/src/swap_device.rs:96
1702024-08-28T00:51:30.454ZINFOSledAgent: Mounting backing filesystems file = sled-agent/src/sled_agent.rs:412 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
1712024-08-28T00:51:30.454ZINFOSledAgent (BackingFs): Processing fmd file = sled-agent/src/backing_fs.rs:126
1722024-08-28T00:51:30.454ZINFOSledAgent (BackingFs): Ensuring dataset oxi_1d8f47b2-ceb5-4768-b5e0-de73f9c6288a/backing/fmd file = sled-agent/src/backing_fs.rs:136
1732024-08-28T00:51:30.460ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1742024-08-28T00:51:30.461ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 62.678656ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "f122e961-2698-49c4-9571-b1566de27f07", "content-length": "124", "date": "Wed, 28 Aug 2024 00:51:29 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "f122e961-2698-49c4-9571-b1566de27f07" } file = clients/ddm-admin-client/src/lib.rs:177
1752024-08-28T00:51:30.525ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1762024-08-28T00:51:30.525ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 160.359712ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "c26b145f-4565-4d7a-b46f-0f27bcc381cc", "content-length": "124", "date": "Wed, 28 Aug 2024 00:51:29 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "c26b145f-4565-4d7a-b46f-0f27bcc381cc" } file = clients/ddm-admin-client/src/lib.rs:177
1772024-08-28T00:51:30.529ZINFOSledAgent (BackingFs): Stopping service svc:/system/fmd:default file = sled-agent/src/backing_fs.rs:170
1782024-08-28T00:51:30.687ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1792024-08-28T00:51:30.688ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 138.967796ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "6e0a8a87-330e-4624-b35a-881a93f1a25b", "content-length": "124", "date": "Wed, 28 Aug 2024 00:51:30 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "6e0a8a87-330e-4624-b35a-881a93f1a25b" } file = clients/ddm-admin-client/src/lib.rs:177
1802024-08-28T00:51:30.749ZINFOSledAgent (BackingFs): Mounting oxi_1d8f47b2-ceb5-4768-b5e0-de73f9c6288a/backing/fmd on /var/fm/fmd file = sled-agent/src/backing_fs.rs:178
1812024-08-28T00:51:30.761ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/rsrc file = sled-agent/src/backing_fs.rs:186
1822024-08-28T00:51:30.761ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/ckpt file = sled-agent/src/backing_fs.rs:186
1832024-08-28T00:51:30.761ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/xprt file = sled-agent/src/backing_fs.rs:186
1842024-08-28T00:51:30.761ZINFOSledAgent (BackingFs): Starting service svc:/system/fmd:default file = sled-agent/src/backing_fs.rs:192
1852024-08-28T00:51:30.828ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1862024-08-28T00:51:30.828ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 206.534799ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "1303440d-1527-47ba-bb9f-a83cf3183652", "content-length": "124", "date": "Wed, 28 Aug 2024 00:51:30 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "1303440d-1527-47ba-bb9f-a83cf3183652" } file = clients/ddm-admin-client/src/lib.rs:177
1872024-08-28T00:51:31.036ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1882024-08-28T00:51:31.037ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 198.821793ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "39aed4ab-1fbe-4766-98a7-9c3e178a3618", "content-length": "124", "date": "Wed, 28 Aug 2024 00:51:30 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "39aed4ab-1fbe-4766-98a7-9c3e178a3618" } file = clients/ddm-admin-client/src/lib.rs:177
1892024-08-28T00:51:31.237ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1902024-08-28T00:51:31.237ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 315.071397ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "21d4027d-4c47-4bd9-bd94-b5f45170a572", "content-length": "124", "date": "Wed, 28 Aug 2024 00:51:30 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "21d4027d-4c47-4bd9-bd94-b5f45170a572" } file = clients/ddm-admin-client/src/lib.rs:177
1912024-08-28T00:51:31.490ZINFOSledAgent: using '[AddrObject { interface: "net0", name: "ll" }, AddrObject { interface: "net1", name: "ll" }]' as data links for xde driver file = illumos-utils/src/opte/illumos.rs:90 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
1922024-08-28T00:51:31.496ZINFOSledAgent (dropshot): listening file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:205 local_addr = [fd00:1122:3344:101::1]:41997 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
1932024-08-28T00:51:31.496ZINFOSledAgent (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = internal-dns/src/resolver.rs:73 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
1942024-08-28T00:51:31.496ZINFOSledAgent (producer-server): starting oximeter metric producer server address = [fd00:1122:3344:101::1]:41997 file = oximeter/producer/src/lib.rs:283 interval = 30s producer_id = 45733be1-aef6-43c8-be71-719893a1feda sled_id = 45733be1-aef6-43c8-be71-719893a1feda
1952024-08-28T00:51:31.497ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 154.590447ms error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
1962024-08-28T00:51:31.498ZINFOSledAgent (VmmReservoirManager): 50% of 102952026112 physical ram = 51476013056 bytes) file = sled-agent/src/vmm_reservoir.rs:267 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
1972024-08-28T00:51:31.498ZINFOSledAgent (VmmReservoirManager): Setting reservoir size to 49090 MiB bytes file = sled-agent/src/vmm_reservoir.rs:275 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
1982024-08-28T00:51:31.499ZINFOSledAgent (ServiceManager): sled agent started file = sled-agent/src/services.rs:915 underlay_address = fd00:1122:3344:101::1
1992024-08-28T00:51:31.499ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = internal-dns/src/resolver.rs:73
2002024-08-28T00:51:31.553ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
2012024-08-28T00:51:31.654ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 297.035164ms error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
2022024-08-28T00:51:31.953ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 614.139737ms error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
2032024-08-28T00:51:32.569ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 2.722315214s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
2042024-08-28T00:51:33.839ZINFOSledAgent (ServiceManager): Setting up dendrite service file = sled-agent/src/services.rs:2634
2052024-08-28T00:51:33.839ZINFOSledAgent (ServiceManager): Setting up lldpd service file = sled-agent/src/services.rs:2852
2062024-08-28T00:51:33.839ZINFOSledAgent (ServiceManager): Setting up MGS service file = sled-agent/src/services.rs:2504
2072024-08-28T00:51:33.839ZINFOSledAgent (ServiceManager): Setting up wicketd service file = sled-agent/src/services.rs:2551
2082024-08-28T00:51:33.839ZINFOSledAgent (ServiceManager): Setting up mgd service file = sled-agent/src/services.rs:2933
2092024-08-28T00:51:33.839ZINFOSledAgent (ServiceManager): Setting up mg-ddm service file = sled-agent/src/services.rs:2975
2102024-08-28T00:51:33.841ZINFOSledAgent (ServiceManager): Setting up tfport service file = sled-agent/src/services.rs:2777
2112024-08-28T00:51:33.842ZINFOSledAgent (ServiceManager): Setting up Simulated SP service file = sled-agent/src/services.rs:2545
2122024-08-28T00:51:33.842ZINFOSledAgent (ServiceManager): Profile for oxz_switch: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService0'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/mgs"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="address" value='[::1]:12225'/> <propval type="astring" name="id" value='9aef38cd-ce09-4fba-b60b-b335261fc177'/> <propval type="astring" name="rack_id" value='75a88ee6-61ab-40f4-a8f1-caf11e1c9852'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/wicketd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="address" value='[::1]:12226'/> <propval type="astring" name="artifact-address" value='[fdb0:18c0:4d0d:9fb2::2]:12227'/> <propval type="astring" name="baseboard-file" value='/opt/oxide/baseboard.json'/> <propval type="astring" name="mgs-address" value='[::1]:12225'/> <propval type="astring" name="nexus-proxy-address" value='[::]:12229'/> <propval type="astring" name="rack-subnet" value='fd00:1122:3344::'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/switch_zone_setup"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="gz_local_link_addr" value='fe80::8:20ff:fe3b:ae94'/> <property type="astring" name="link_local_links"> <astring_list> <value_node value="tfportrear0_0" /> <value_node value="oxBootstrap0" /> </astring_list> </property> <propval type="astring" name="bootstrap_addr" value='fdb0:18c0:4d0d:9fb2::2'/> <propval type="astring" name="bootstrap_vnic" value='oxBootstrap0'/> <propval type="astring" name="baseboard_info" value='{"type":"pc","identifier":"buskin","model":"i86pc"}'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/dendrite"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="sled_id" value='45733be1-aef6-43c8-be71-719893a1feda'/> <propval type="astring" name="rack_id" value='75a88ee6-61ab-40f4-a8f1-caf11e1c9852'/> <propval type="astring" name="address" value='[::1]:12224'/> <propval type="astring" name="front_ports" value='1'/> <propval type="astring" name="rear_ports" value='1'/> <propval type="astring" name="port_config" value='/opt/oxide/dendrite/misc/softnpu_single_sled_config.toml'/> <propval type="astring" name="mgmt" value='uds'/> <propval type="astring" name="uds_path" value='/opt/softnpu/stuff'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/tfport"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="host" value='[::1]'/> <propval type="astring" name="port" value='12224'/> <propval type="astring" name="flags" value='--sync-only'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/lldpd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="board_rev" value='softnpu_front_1_rear_1'/> <propval type="astring" name="scrimlet_id" value='buskin'/> <propval type="astring" name="scrimlet_model" value='i86pc'/> <propval type="astring" name="address" value='[::1]:12230'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/pumpkind"> </service> <service version="1" type="service" name="oxide/mgd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="sled_uuid" value='45733be1-aef6-43c8-be71-719893a1feda'/> <propval type="astring" name="rack_uuid" value='75a88ee6-61ab-40f4-a8f1-caf11e1c9852'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/mg-ddm"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="mode" value='transit'/> <propval type="astring" name="dendrite" value='true'/> <propval type="astring" name="sled_uuid" value='45733be1-aef6-43c8-be71-719893a1feda'/> <propval type="astring" name="rack_uuid" value='75a88ee6-61ab-40f4-a8f1-caf11e1c9852'/> <propval type="astring" name="interfaces" value='tfportrear0_0/ll'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/uplink"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
2132024-08-28T00:51:33.842ZINFOSledAgent (ServiceManager): Booting oxz_switch zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_switch
2142024-08-28T00:51:34.081ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 49.672407ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2152024-08-28T00:51:34.260ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 101.712205ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2162024-08-28T00:51:34.440ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 284.737417ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2172024-08-28T00:51:34.793ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 352.226955ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2182024-08-28T00:51:35.168ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 1.14250674s file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2192024-08-28T00:51:35.293ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 3.548922915s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
2202024-08-28T00:51:36.333ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 955.437044ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2212024-08-28T00:51:37.461ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 680.111974ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2222024-08-28T00:51:38.361ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 722.860779ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2232024-08-28T00:51:38.844ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 5.625757293s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
2242024-08-28T00:51:39.565ZINFOSledAgent (dropshot (SledAgent)): listening file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:205 local_addr = [fd00:1122:3344:101::1]:12345
2252024-08-28T00:51:39.565ZINFOSledAgent (SledAgentMain): Sled Agent loaded; recording configuration file = sled-agent/src/bootstrap/server.rs:429
2262024-08-28T00:51:39.565ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
2272024-08-28T00:51:39.565ZINFOSledAgent (SledAgentMain): Writing ledger to /pool/int/1d8f47b2-ceb5-4768-b5e0-de73f9c6288a/config/.sled-agent-request.json.tmp file = common/src/ledger.rs:199
2282024-08-28T00:51:39.566ZINFOSledAgent (SledAgentMain): Writing ledger to /pool/int/2701509e-5560-4f63-bb12-b820fcc5e4cb/config/.sled-agent-request.json.tmp file = common/src/ledger.rs:199
2292024-08-28T00:51:39.566ZINFOSledAgent (HardwareMonitor): Sled Agent Started file = sled-agent/src/hardware_monitor.rs:138
2302024-08-28T00:51:39.566ZINFOSledAgent (HardwareMonitor): Checking current full hardware snapshot disks = {} file = sled-agent/src/hardware_monitor.rs:271 underlay_network_info = Some((fd00:1122:3344:101::2, Some(RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56 }, infra_ip_first: 10.151.1.174, infra_ip_last: 10.151.1.174, ports: [PortConfigV2 { routes: [RouteConfig { destination: V4(Ipv4Net { addr: 0.0.0.0, width: 0 }), nexthop: 10.151.1.100, vlan_id: None, local_pref: None }], addresses: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: None, bgp_peers: [], autoneg: false, lldp: None }], bgp: [], bfd: [] })))
2312024-08-28T00:51:39.566ZINFOSledAgent (ServiceManager): Ensuring scrimlet services (enabling services) file = sled-agent/src/services.rs:3861
2322024-08-28T00:51:39.566ZINFOSledAgent (SledAgentSprocketsServer): Connection closed file = sled-agent/src/bootstrap/sprockets_server.rs:73 remote_addr = [fdb0:18c0:4d0d:9fb2::1]:42213
2332024-08-28T00:51:39.803ZINFOSledAgent (BootstrapAgentRssHandler): Peer agent initialized file = sled-agent/src/bootstrap/rss_handle.rs:108 peer = [fdb0:18c0:4d0d:9fb2::1]:12346
2342024-08-28T00:51:39.803ZINFOSledAgent (BootstrapAgentRssHandler): Initialized sled agent file = sled-agent/src/bootstrap/rss_handle.rs:232 target_sled = [fdb0:18c0:4d0d:9fb2::1]:12346
2352024-08-28T00:51:39.803ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
2362024-08-28T00:51:39.803ZINFOSledAgent (RSS): No ledger in /pool/int/1d8f47b2-ceb5-4768-b5e0-de73f9c6288a/config/rss-service-plan-v4.json file = common/src/ledger.rs:188
2372024-08-28T00:51:39.803ZINFOSledAgent (RSS): No ledger in /pool/int/2701509e-5560-4f63-bb12-b820fcc5e4cb/config/rss-service-plan-v4.json file = common/src/ledger.rs:188
2382024-08-28T00:51:39.803ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
2392024-08-28T00:51:39.804ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
2402024-08-28T00:51:39.804ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
2412024-08-28T00:51:39.809ZINFOSledAgent (ServiceManager): Re-enabling running switch zone (new address) file = sled-agent/src/services.rs:4182 new = [fd00:1122:3344:101::2, ::1] old = [::1]
2422024-08-28T00:51:39.809ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::2 exists file = sled-agent/src/services.rs:4198
2432024-08-28T00:51:39.809ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:101::2, prefix: 64 })) file = illumos-utils/src/running_zone.rs:534 zone = oxz_switch
2442024-08-28T00:51:40.054ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:56825
2452024-08-28T00:51:40.055ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
2462024-08-28T00:51:40.055ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 302 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:56825 req_id = e3b7c5b0-857c-44a5-8215-3e4f02f36b8e response_code = 200 uri = /inventory
2472024-08-28T00:51:40.085ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:51835
2482024-08-28T00:51:40.085ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 37 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:51835 req_id = 10c25601-8511-4f50-a4b0-d13fb51d14c5 response_code = 200 uri = /sled-role
2492024-08-28T00:51:40.086ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
2502024-08-28T00:51:40.086ZINFOSledAgent (RSS): Writing ledger to /pool/int/1d8f47b2-ceb5-4768-b5e0-de73f9c6288a/config/.rss-service-plan-v4.json.tmp file = common/src/ledger.rs:199
2512024-08-28T00:51:40.087ZINFOSledAgent (RSS): Writing ledger to /pool/int/2701509e-5560-4f63-bb12-b820fcc5e4cb/config/.rss-service-plan-v4.json.tmp file = common/src/ledger.rs:199
2522024-08-28T00:51:40.087ZINFOSledAgent (RSS): Service plan written to storage file = sled-agent/src/rack_setup/plan/service.rs:998
2532024-08-28T00:51:40.119ZINFOSledAgent (RSS): attempting to set up sled's storage: OmicronPhysicalDisksConfig { generation: Generation(1), disks: [OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, id: 3d6fd55b-46e6-4396-a673-d192ddcf79be, pool_id: 2deff32e-1106-452e-a0e4-7f5633a4a1c4 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, id: e6ca764f-526a-4200-922f-48e1852ff2dd, pool_id: 903e5bdb-4fa3-466b-9cc6-cb6c1205c20b (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, id: c9ac2379-ef4e-45b1-a5be-44d617891ca5, pool_id: 8c0884cf-f16d-4d49-a677-c20655479168 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, id: 8b5005ee-12f3-484d-b473-81bc830fccec, pool_id: 39cc409d-9820-46b7-b64f-11f4d431b612 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, id: 65d13c3f-1260-418b-8658-9ea0b1f42538, pool_id: 753cff17-fbdf-48d1-960a-a1fe6d8c01ed (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, id: 5a071ba9-ec1e-4dc8-b44b-c78d76d0c7e3, pool_id: fe61a674-6b99-47f9-b238-13f6d9a31093 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, id: 26c7c76b-3fe5-4a58-9b5a-a83f8e5db0fd, pool_id: 9b7eedae-40f6-4e96-bd9d-f89d715d8ad7 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, id: 53551ca9-b818-4f21-9e1e-53e71a5b4615, pool_id: 1a002b08-6021-4882-a443-e176f8f87821 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, id: d7b68149-f4f8-46cb-8b2e-5befbf6e6bf2, pool_id: 2936c097-160d-4a3f-a5e8-a1c6ffbf1edb (zpool) }] } file = sled-agent/src/rack_setup/service.rs:355 sled_address = [fd00:1122:3344:101::1]:12345
2542024-08-28T00:51:40.119ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:51582
2552024-08-28T00:51:40.119ZINFOSledAgent: physical disks ensure file = sled-agent/src/sled_agent.rs:830 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
2562024-08-28T00:51:40.119ZINFOSledAgent (StorageManager): Received OmicronPhysicalDisksEnsure { config: OmicronPhysicalDisksConfig { generation: Generation(1), disks: [OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, id: 3d6fd55b-46e6-4396-a673-d192ddcf79be, pool_id: 2deff32e-1106-452e-a0e4-7f5633a4a1c4 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, id: e6ca764f-526a-4200-922f-48e1852ff2dd, pool_id: 903e5bdb-4fa3-466b-9cc6-cb6c1205c20b (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, id: c9ac2379-ef4e-45b1-a5be-44d617891ca5, pool_id: 8c0884cf-f16d-4d49-a677-c20655479168 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, id: 8b5005ee-12f3-484d-b473-81bc830fccec, pool_id: 39cc409d-9820-46b7-b64f-11f4d431b612 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, id: 65d13c3f-1260-418b-8658-9ea0b1f42538, pool_id: 753cff17-fbdf-48d1-960a-a1fe6d8c01ed (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, id: 5a071ba9-ec1e-4dc8-b44b-c78d76d0c7e3, pool_id: fe61a674-6b99-47f9-b238-13f6d9a31093 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, id: 26c7c76b-3fe5-4a58-9b5a-a83f8e5db0fd, pool_id: 9b7eedae-40f6-4e96-bd9d-f89d715d8ad7 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, id: 53551ca9-b818-4f21-9e1e-53e71a5b4615, pool_id: 1a002b08-6021-4882-a443-e176f8f87821 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, id: d7b68149-f4f8-46cb-8b2e-5befbf6e6bf2, pool_id: 2936c097-160d-4a3f-a5e8-a1c6ffbf1edb (zpool) }] }, tx: ... } file = sled-storage/src/manager.rs:406
2572024-08-28T00:51:40.119ZINFOSledAgent (StorageManager): No ledger in /pool/int/1d8f47b2-ceb5-4768-b5e0-de73f9c6288a/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = omicron_physical_disks_ensure
2582024-08-28T00:51:40.119ZINFOSledAgent (StorageManager): No ledger in /pool/int/2701509e-5560-4f63-bb12-b820fcc5e4cb/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = omicron_physical_disks_ensure
2592024-08-28T00:51:40.119ZINFOSledAgent (StorageManager): No previously-stored 'requested disks', creating new ledger file = sled-storage/src/manager.rs:645 request = omicron_physical_disks_ensure
2602024-08-28T00:51:40.119ZINFOSledAgent (StorageResources): Synchronizing disk managment file = sled-storage/src/resources.rs:302
2612024-08-28T00:51:40.119ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:342
2622024-08-28T00:51:40.119ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
2632024-08-28T00:51:40.119ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } path = /scratch/u2_0.vdev
2642024-08-28T00:51:40.264ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_0.vdev file = sled-hardware/src/disk.rs:360
2652024-08-28T00:51:40.264ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = 2deff32e-1106-452e-a0e4-7f5633a4a1c4 (zpool)
2662024-08-28T00:51:40.743ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/45733be1-aef6-43c8-be71-719893a1feda): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/probe_manager.rs:230 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
2672024-08-28T00:51:41.443ZINFOSledAgent (VmmReservoirManager): Finished setting reservoir size to 49090 MiB bytes file = sled-agent/src/vmm_reservoir.rs:279 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
2682024-08-28T00:51:41.453ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:307 zpool = ZpoolName { id: 2deff32e-1106-452e-a0e4-7f5633a4a1c4 (zpool), kind: External }
2692024-08-28T00:51:41.461ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:351
2702024-08-28T00:51:41.462ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:353
2712024-08-28T00:51:41.462ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:357
2722024-08-28T00:51:41.462ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:359
2732024-08-28T00:51:41.462ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key")
2742024-08-28T00:51:41.462ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key")
2752024-08-28T00:51:41.462ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:371
2762024-08-28T00:51:41.497ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
2772024-08-28T00:51:41.505ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_2deff32e-1106-452e-a0e4-7f5633a4a1c4/crypt/zone file = sled-storage/src/dataset.rs:414
2782024-08-28T00:51:41.631ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_2deff32e-1106-452e-a0e4-7f5633a4a1c4 file = sled-storage/src/dataset.rs:499
2792024-08-28T00:51:41.639ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:440
2802024-08-28T00:51:41.639ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:357
2812024-08-28T00:51:41.639ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:342
2822024-08-28T00:51:41.639ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
2832024-08-28T00:51:41.639ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } path = /scratch/u2_1.vdev
2842024-08-28T00:51:41.649ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_1.vdev file = sled-hardware/src/disk.rs:360
2852024-08-28T00:51:41.649ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = 903e5bdb-4fa3-466b-9cc6-cb6c1205c20b (zpool)
2862024-08-28T00:51:41.739ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:307 zpool = ZpoolName { id: 903e5bdb-4fa3-466b-9cc6-cb6c1205c20b (zpool), kind: External }
2872024-08-28T00:51:41.748ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:351
2882024-08-28T00:51:41.748ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:353
2892024-08-28T00:51:41.748ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:357
2902024-08-28T00:51:41.748ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:359
2912024-08-28T00:51:41.748ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key")
2922024-08-28T00:51:41.748ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key")
2932024-08-28T00:51:41.748ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:371
2942024-08-28T00:51:41.752ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/45733be1-aef6-43c8-be71-719893a1feda): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/probe_manager.rs:230 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
2952024-08-28T00:51:41.783ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
2962024-08-28T00:51:41.791ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_903e5bdb-4fa3-466b-9cc6-cb6c1205c20b/crypt/zone file = sled-storage/src/dataset.rs:414
2972024-08-28T00:51:41.913ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_903e5bdb-4fa3-466b-9cc6-cb6c1205c20b file = sled-storage/src/dataset.rs:499
2982024-08-28T00:51:41.921ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:440
2992024-08-28T00:51:41.921ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:357
3002024-08-28T00:51:41.921ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:342
3012024-08-28T00:51:41.921ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3022024-08-28T00:51:41.921ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } path = /scratch/u2_2.vdev
3032024-08-28T00:51:41.930ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_2.vdev file = sled-hardware/src/disk.rs:360
3042024-08-28T00:51:41.930ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = 8c0884cf-f16d-4d49-a677-c20655479168 (zpool)
3052024-08-28T00:51:42.022ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/dataset.rs:307 zpool = ZpoolName { id: 8c0884cf-f16d-4d49-a677-c20655479168 (zpool), kind: External }
3062024-08-28T00:51:42.030ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/dataset.rs:351
3072024-08-28T00:51:42.030ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:353
3082024-08-28T00:51:42.030ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:357
3092024-08-28T00:51:42.030ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:359
3102024-08-28T00:51:42.030ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key")
3112024-08-28T00:51:42.030ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key")
3122024-08-28T00:51:42.030ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:371
3132024-08-28T00:51:42.066ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3142024-08-28T00:51:42.074ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_8c0884cf-f16d-4d49-a677-c20655479168/crypt/zone file = sled-storage/src/dataset.rs:414
3152024-08-28T00:51:42.202ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_8c0884cf-f16d-4d49-a677-c20655479168 file = sled-storage/src/dataset.rs:499
3162024-08-28T00:51:42.210ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:440
3172024-08-28T00:51:42.210ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:357
3182024-08-28T00:51:42.210ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:342
3192024-08-28T00:51:42.210ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3202024-08-28T00:51:42.210ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } path = /scratch/u2_3.vdev
3212024-08-28T00:51:42.219ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_3.vdev file = sled-hardware/src/disk.rs:360
3222024-08-28T00:51:42.219ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = 39cc409d-9820-46b7-b64f-11f4d431b612 (zpool)
3232024-08-28T00:51:42.316ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/dataset.rs:307 zpool = ZpoolName { id: 39cc409d-9820-46b7-b64f-11f4d431b612 (zpool), kind: External }
3242024-08-28T00:51:42.324ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/dataset.rs:351
3252024-08-28T00:51:42.324ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:353
3262024-08-28T00:51:42.324ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:357
3272024-08-28T00:51:42.324ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:359
3282024-08-28T00:51:42.324ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key")
3292024-08-28T00:51:42.324ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key")
3302024-08-28T00:51:42.324ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:371
3312024-08-28T00:51:42.358ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3322024-08-28T00:51:42.366ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_39cc409d-9820-46b7-b64f-11f4d431b612/crypt/zone file = sled-storage/src/dataset.rs:414
3332024-08-28T00:51:42.488ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_39cc409d-9820-46b7-b64f-11f4d431b612 file = sled-storage/src/dataset.rs:499
3342024-08-28T00:51:42.496ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:440
3352024-08-28T00:51:42.496ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:357
3362024-08-28T00:51:42.496ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:342
3372024-08-28T00:51:42.496ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3382024-08-28T00:51:42.496ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } path = /scratch/u2_4.vdev
3392024-08-28T00:51:42.505ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_4.vdev file = sled-hardware/src/disk.rs:360
3402024-08-28T00:51:42.505ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = 753cff17-fbdf-48d1-960a-a1fe6d8c01ed (zpool)
3412024-08-28T00:51:42.600ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/dataset.rs:307 zpool = ZpoolName { id: 753cff17-fbdf-48d1-960a-a1fe6d8c01ed (zpool), kind: External }
3422024-08-28T00:51:42.608ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/dataset.rs:351
3432024-08-28T00:51:42.608ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:353
3442024-08-28T00:51:42.608ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:357
3452024-08-28T00:51:42.608ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:359
3462024-08-28T00:51:42.608ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key")
3472024-08-28T00:51:42.608ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key")
3482024-08-28T00:51:42.608ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:371
3492024-08-28T00:51:42.644ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3502024-08-28T00:51:42.651ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_753cff17-fbdf-48d1-960a-a1fe6d8c01ed/crypt/zone file = sled-storage/src/dataset.rs:414
3512024-08-28T00:51:42.754ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/45733be1-aef6-43c8-be71-719893a1feda): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/probe_manager.rs:230 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
3522024-08-28T00:51:42.775ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_753cff17-fbdf-48d1-960a-a1fe6d8c01ed file = sled-storage/src/dataset.rs:499
3532024-08-28T00:51:42.783ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:440
3542024-08-28T00:51:42.783ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:357
3552024-08-28T00:51:42.783ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:342
3562024-08-28T00:51:42.783ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3572024-08-28T00:51:42.783ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } path = /scratch/u2_5.vdev
3582024-08-28T00:51:42.793ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_5.vdev file = sled-hardware/src/disk.rs:360
3592024-08-28T00:51:42.793ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = fe61a674-6b99-47f9-b238-13f6d9a31093 (zpool)
3602024-08-28T00:51:42.893ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/dataset.rs:307 zpool = ZpoolName { id: fe61a674-6b99-47f9-b238-13f6d9a31093 (zpool), kind: External }
3612024-08-28T00:51:42.901ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/dataset.rs:351
3622024-08-28T00:51:42.901ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:353
3632024-08-28T00:51:42.901ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:357
3642024-08-28T00:51:42.901ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:359
3652024-08-28T00:51:42.901ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key")
3662024-08-28T00:51:42.901ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key")
3672024-08-28T00:51:42.901ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:371
3682024-08-28T00:51:42.935ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3692024-08-28T00:51:42.943ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_fe61a674-6b99-47f9-b238-13f6d9a31093/crypt/zone file = sled-storage/src/dataset.rs:414
3702024-08-28T00:51:43.068ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_fe61a674-6b99-47f9-b238-13f6d9a31093 file = sled-storage/src/dataset.rs:499
3712024-08-28T00:51:43.076ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:440
3722024-08-28T00:51:43.076ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:357
3732024-08-28T00:51:43.076ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:342
3742024-08-28T00:51:43.076ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3752024-08-28T00:51:43.077ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } path = /scratch/u2_6.vdev
3762024-08-28T00:51:43.086ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_6.vdev file = sled-hardware/src/disk.rs:360
3772024-08-28T00:51:43.086ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = 9b7eedae-40f6-4e96-bd9d-f89d715d8ad7 (zpool)
3782024-08-28T00:51:43.180ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/dataset.rs:307 zpool = ZpoolName { id: 9b7eedae-40f6-4e96-bd9d-f89d715d8ad7 (zpool), kind: External }
3792024-08-28T00:51:43.188ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/dataset.rs:351
3802024-08-28T00:51:43.188ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:353
3812024-08-28T00:51:43.188ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:357
3822024-08-28T00:51:43.188ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:359
3832024-08-28T00:51:43.188ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key")
3842024-08-28T00:51:43.188ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key")
3852024-08-28T00:51:43.188ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:371
3862024-08-28T00:51:43.225ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3872024-08-28T00:51:43.233ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_9b7eedae-40f6-4e96-bd9d-f89d715d8ad7/crypt/zone file = sled-storage/src/dataset.rs:414
3882024-08-28T00:51:43.354ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_9b7eedae-40f6-4e96-bd9d-f89d715d8ad7 file = sled-storage/src/dataset.rs:499
3892024-08-28T00:51:43.362ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:440
3902024-08-28T00:51:43.362ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:357
3912024-08-28T00:51:43.362ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:342
3922024-08-28T00:51:43.362ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3932024-08-28T00:51:43.362ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } path = /scratch/u2_7.vdev
3942024-08-28T00:51:43.371ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_7.vdev file = sled-hardware/src/disk.rs:360
3952024-08-28T00:51:43.371ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = 1a002b08-6021-4882-a443-e176f8f87821 (zpool)
3962024-08-28T00:51:43.468ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/dataset.rs:307 zpool = ZpoolName { id: 1a002b08-6021-4882-a443-e176f8f87821 (zpool), kind: External }
3972024-08-28T00:51:43.476ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/dataset.rs:351
3982024-08-28T00:51:43.476ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:353
3992024-08-28T00:51:43.476ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:357
4002024-08-28T00:51:43.476ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:359
4012024-08-28T00:51:43.476ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key")
4022024-08-28T00:51:43.476ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key")
4032024-08-28T00:51:43.476ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:371
4042024-08-28T00:51:43.514ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
4052024-08-28T00:51:43.523ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_1a002b08-6021-4882-a443-e176f8f87821/crypt/zone file = sled-storage/src/dataset.rs:414
4062024-08-28T00:51:43.650ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_1a002b08-6021-4882-a443-e176f8f87821 file = sled-storage/src/dataset.rs:499
4072024-08-28T00:51:43.658ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:440
4082024-08-28T00:51:43.658ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:357
4092024-08-28T00:51:43.658ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:342
4102024-08-28T00:51:43.658ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
4112024-08-28T00:51:43.658ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } path = /scratch/u2_8.vdev
4122024-08-28T00:51:43.667ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_8.vdev file = sled-hardware/src/disk.rs:360
4132024-08-28T00:51:43.667ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = 2936c097-160d-4a3f-a5e8-a1c6ffbf1edb (zpool)
4142024-08-28T00:51:43.762ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/dataset.rs:307 zpool = ZpoolName { id: 2936c097-160d-4a3f-a5e8-a1c6ffbf1edb (zpool), kind: External }
4152024-08-28T00:51:43.770ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/dataset.rs:351
4162024-08-28T00:51:43.770ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:353
4172024-08-28T00:51:43.770ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:357
4182024-08-28T00:51:43.770ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:359
4192024-08-28T00:51:43.770ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key")
4202024-08-28T00:51:43.770ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key")
4212024-08-28T00:51:43.770ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:371
4222024-08-28T00:51:43.774ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/45733be1-aef6-43c8-be71-719893a1feda): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/probe_manager.rs:230 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
4232024-08-28T00:51:43.806ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
4242024-08-28T00:51:43.815ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_2936c097-160d-4a3f-a5e8-a1c6ffbf1edb/crypt/zone file = sled-storage/src/dataset.rs:414
4252024-08-28T00:51:43.937ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_2936c097-160d-4a3f-a5e8-a1c6ffbf1edb file = sled-storage/src/dataset.rs:499
4262024-08-28T00:51:43.945ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:440
4272024-08-28T00:51:43.945ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:357
4282024-08-28T00:51:43.945ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/1d8f47b2-ceb5-4768-b5e0-de73f9c6288a/config/.omicron-physical-disks.json.tmp file = common/src/ledger.rs:199 request = omicron_physical_disks_ensure
4292024-08-28T00:51:43.945ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 1d8f47b2-ceb5-4768-b5e0-de73f9c6288a (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 2701509e-5560-4f63-bb12-b820fcc5e4cb (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 2deff32e-1106-452e-a0e4-7f5633a4a1c4 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 903e5bdb-4fa3-466b-9cc6-cb6c1205c20b (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 8c0884cf-f16d-4d49-a677-c20655479168 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 39cc409d-9820-46b7-b64f-11f4d431b612 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 753cff17-fbdf-48d1-960a-a1fe6d8c01ed (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: fe61a674-6b99-47f9-b238-13f6d9a31093 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 9b7eedae-40f6-4e96-bd9d-f89d715d8ad7 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 1a002b08-6021-4882-a443-e176f8f87821 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 2936c097-160d-4a3f-a5e8-a1c6ffbf1edb (zpool), kind: External } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
4302024-08-28T00:51:43.945ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
4312024-08-28T00:51:43.945ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
4322024-08-28T00:51:43.945ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/2701509e-5560-4f63-bb12-b820fcc5e4cb/config/.omicron-physical-disks.json.tmp file = common/src/ledger.rs:199 request = omicron_physical_disks_ensure
4332024-08-28T00:51:43.946ZINFOSledAgent: physical disks ensure: Updated storage file = sled-agent/src/sled_agent.rs:834 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
4342024-08-28T00:51:43.946ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
4352024-08-28T00:51:43.946ZINFOSledAgent: physical disks ensure: Propagating new generation of disks file = sled-agent/src/sled_agent.rs:860 generation = Generation(1) sled_id = 45733be1-aef6-43c8-be71-719893a1feda
4362024-08-28T00:51:43.946ZINFOSledAgent: physical disks ensure: Updated storage monitor file = sled-agent/src/sled_agent.rs:865 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
4372024-08-28T00:51:43.946ZINFOSledAgent: physical disks ensure: Updated zone bundler file = sled-agent/src/sled_agent.rs:870 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
4382024-08-28T00:51:43.946ZINFOSledAgent (ProbeManager): use_only_these_disks: Processing new request file = sled-agent/src/probe_manager.rs:121 gen = Generation(1) sled_id = 45733be1-aef6-43c8-be71-719893a1feda
4392024-08-28T00:51:43.946ZINFOSledAgent: physical disks ensure: Updated probes file = sled-agent/src/sled_agent.rs:876 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
4402024-08-28T00:51:43.946ZINFOSledAgent (InstanceManager): use_only_these_disks: Processing new request file = sled-agent/src/instance_manager.rs:803 gen = Generation(1)
4412024-08-28T00:51:43.946ZINFOSledAgent: physical disks ensure: Updated instances file = sled-agent/src/sled_agent.rs:881 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
4422024-08-28T00:51:43.946ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3827144 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:51582 req_id = 76f4fe86-1cb2-4c38-b27d-0357a775f99d response_code = 200 uri = /omicron-physical-disks
4432024-08-28T00:51:43.978ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(2), zones: [OmicronZoneConfig { id: 7ad61349-3006-4e30-bd4f-4104affaacb7, underlay_address: fd00:1122:3344:1::1, filesystem_pool: Some(ZpoolName { id: 2deff32e-1106-452e-a0e4-7f5633a4a1c4 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 2deff32e-1106-452e-a0e4-7f5633a4a1c4 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 4dfcfd32-ab45-42c7-9c2f-4d836b875b6a, underlay_address: fd00:1122:3344:2::1, filesystem_pool: Some(ZpoolName { id: 903e5bdb-4fa3-466b-9cc6-cb6c1205c20b (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 903e5bdb-4fa3-466b-9cc6-cb6c1205c20b (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 09ed9686-d85a-429a-baae-bf2b96abe530, underlay_address: fd00:1122:3344:3::1, filesystem_pool: Some(ZpoolName { id: 8c0884cf-f16d-4d49-a677-c20655479168 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8c0884cf-f16d-4d49-a677-c20655479168 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }] } file = sled-agent/src/rack_setup/service.rs:438 sled_address = [fd00:1122:3344:101::1]:12345
4442024-08-28T00:51:43.978ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:50918
4452024-08-28T00:51:43.978ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 7ad61349-3006-4e30-bd4f-4104affaacb7, dataset_name: DatasetName { pool_name: ZpoolName { id: 2deff32e-1106-452e-a0e4-7f5633a4a1c4 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:406
4462024-08-28T00:51:43.978ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 7ad61349-3006-4e30-bd4f-4104affaacb7, dataset_name: DatasetName { pool_name: ZpoolName { id: 2deff32e-1106-452e-a0e4-7f5633a4a1c4 (zpool), kind: External }, kind: InternalDns }, responder: ... }
4472024-08-28T00:51:44.029ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 4dfcfd32-ab45-42c7-9c2f-4d836b875b6a, dataset_name: DatasetName { pool_name: ZpoolName { id: 903e5bdb-4fa3-466b-9cc6-cb6c1205c20b (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:406
4482024-08-28T00:51:44.029ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 4dfcfd32-ab45-42c7-9c2f-4d836b875b6a, dataset_name: DatasetName { pool_name: ZpoolName { id: 903e5bdb-4fa3-466b-9cc6-cb6c1205c20b (zpool), kind: External }, kind: InternalDns }, responder: ... }
4492024-08-28T00:51:44.081ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 09ed9686-d85a-429a-baae-bf2b96abe530, dataset_name: DatasetName { pool_name: ZpoolName { id: 8c0884cf-f16d-4d49-a677-c20655479168 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:406
4502024-08-28T00:51:44.081ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 09ed9686-d85a-429a-baae-bf2b96abe530, dataset_name: DatasetName { pool_name: ZpoolName { id: 8c0884cf-f16d-4d49-a677-c20655479168 (zpool), kind: External }, kind: InternalDns }, responder: ... }
4512024-08-28T00:51:44.131ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
4522024-08-28T00:51:44.131ZINFOSledAgent (ServiceManager): No ledger in /pool/int/1d8f47b2-ceb5-4768-b5e0-de73f9c6288a/config/omicron-zones.json file = common/src/ledger.rs:188
4532024-08-28T00:51:44.131ZINFOSledAgent (ServiceManager): No ledger in /pool/int/2701509e-5560-4f63-bb12-b820fcc5e4cb/config/omicron-zones.json file = common/src/ledger.rs:188
4542024-08-28T00:51:44.131ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
4552024-08-28T00:51:44.176ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
4562024-08-28T00:51:44.176ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
4572024-08-28T00:51:44.176ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
4582024-08-28T00:51:44.228ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_4dfcfd32-ab45-42c7-9c2f-4d836b875b6a file = illumos-utils/src/zone.rs:316
4592024-08-28T00:51:44.232ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_7ad61349-3006-4e30-bd4f-4104affaacb7 file = illumos-utils/src/zone.rs:316
4602024-08-28T00:51:44.236ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_09ed9686-d85a-429a-baae-bf2b96abe530 file = illumos-utils/src/zone.rs:316
4612024-08-28T00:51:44.248ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_4dfcfd32-ab45-42c7-9c2f-4d836b875b6a file = illumos-utils/src/zone.rs:354
4622024-08-28T00:51:44.252ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_7ad61349-3006-4e30-bd4f-4104affaacb7 file = illumos-utils/src/zone.rs:354
4632024-08-28T00:51:44.256ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_09ed9686-d85a-429a-baae-bf2b96abe530 file = illumos-utils/src/zone.rs:354
4642024-08-28T00:51:44.473ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 23.215363009s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
4652024-08-28T00:51:44.776ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/45733be1-aef6-43c8-be71-719893a1feda): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/probe_manager.rs:230 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
4662024-08-28T00:51:45.731ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_09ed9686-d85a-429a-baae-bf2b96abe530: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService3'/> <propval type="astring" name="gateway" value='fd00:1122:3344:3::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:3::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:3::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:3::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
4672024-08-28T00:51:45.731ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:3::, width: 64 }
4682024-08-28T00:51:46.209ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_7ad61349-3006-4e30-bd4f-4104affaacb7: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService2'/> <propval type="astring" name="gateway" value='fd00:1122:3344:1::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:1::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:1::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:1::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
4692024-08-28T00:51:46.209ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_09ed9686-d85a-429a-baae-bf2b96abe530 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_09ed9686-d85a-429a-baae-bf2b96abe530
4702024-08-28T00:51:46.212ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:1::, width: 64 }
4712024-08-28T00:51:46.692ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_4dfcfd32-ab45-42c7-9c2f-4d836b875b6a: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService1'/> <propval type="astring" name="gateway" value='fd00:1122:3344:2::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:2::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:2::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:2::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
4722024-08-28T00:51:46.692ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_7ad61349-3006-4e30-bd4f-4104affaacb7 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_7ad61349-3006-4e30-bd4f-4104affaacb7
4732024-08-28T00:51:46.707ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_09ed9686-d85a-429a-baae-bf2b96abe530") failed: Property not found. retry in 41.341139ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_09ed9686-d85a-429a-baae-bf2b96abe530
4742024-08-28T00:51:46.707ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:2::, width: 64 }
4752024-08-28T00:51:46.707ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_4dfcfd32-ab45-42c7-9c2f-4d836b875b6a zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_4dfcfd32-ab45-42c7-9c2f-4d836b875b6a
4762024-08-28T00:51:46.758ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_09ed9686-d85a-429a-baae-bf2b96abe530") failed: Property not found. retry in 106.215712ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_09ed9686-d85a-429a-baae-bf2b96abe530
4772024-08-28T00:51:46.823ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_7ad61349-3006-4e30-bd4f-4104affaacb7") failed: Property not found. retry in 41.376844ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_7ad61349-3006-4e30-bd4f-4104affaacb7
4782024-08-28T00:51:46.830ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_4dfcfd32-ab45-42c7-9c2f-4d836b875b6a") failed: Property not found. retry in 29.19987ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_4dfcfd32-ab45-42c7-9c2f-4d836b875b6a
4792024-08-28T00:51:46.866ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_4dfcfd32-ab45-42c7-9c2f-4d836b875b6a") failed: Property not found. retry in 127.494036ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_4dfcfd32-ab45-42c7-9c2f-4d836b875b6a
4802024-08-28T00:51:46.873ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_7ad61349-3006-4e30-bd4f-4104affaacb7") failed: Property not found. retry in 128.779018ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_7ad61349-3006-4e30-bd4f-4104affaacb7
4812024-08-28T00:51:46.880ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_09ed9686-d85a-429a-baae-bf2b96abe530") failed: Property not found. retry in 139.805352ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_09ed9686-d85a-429a-baae-bf2b96abe530
4822024-08-28T00:51:47.029ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_4dfcfd32-ab45-42c7-9c2f-4d836b875b6a") failed: Property not found. retry in 210.098241ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_4dfcfd32-ab45-42c7-9c2f-4d836b875b6a
4832024-08-28T00:51:47.115ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_7ad61349-3006-4e30-bd4f-4104affaacb7") failed: Property not found. retry in 154.686822ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_7ad61349-3006-4e30-bd4f-4104affaacb7
4842024-08-28T00:51:47.140ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_09ed9686-d85a-429a-baae-bf2b96abe530") failed: Property not found. retry in 496.925976ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_09ed9686-d85a-429a-baae-bf2b96abe530
4852024-08-28T00:51:47.271ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_4dfcfd32-ab45-42c7-9c2f-4d836b875b6a") failed: Property not found. retry in 282.740644ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_4dfcfd32-ab45-42c7-9c2f-4d836b875b6a
4862024-08-28T00:51:47.287ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_7ad61349-3006-4e30-bd4f-4104affaacb7") failed: Property not found. retry in 356.723693ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_7ad61349-3006-4e30-bd4f-4104affaacb7
4872024-08-28T00:51:47.565ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_4dfcfd32-ab45-42c7-9c2f-4d836b875b6a") failed: Property not found. retry in 628.974943ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_4dfcfd32-ab45-42c7-9c2f-4d836b875b6a
4882024-08-28T00:51:47.646ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_09ed9686-d85a-429a-baae-bf2b96abe530") failed: Property not found. retry in 410.459287ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_09ed9686-d85a-429a-baae-bf2b96abe530
4892024-08-28T00:51:47.653ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_7ad61349-3006-4e30-bd4f-4104affaacb7") failed: Property not found. retry in 482.574651ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_7ad61349-3006-4e30-bd4f-4104affaacb7
4902024-08-28T00:51:48.068ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_09ed9686-d85a-429a-baae-bf2b96abe530") failed: Property not found. retry in 915.251491ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_09ed9686-d85a-429a-baae-bf2b96abe530
4912024-08-28T00:51:48.149ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_7ad61349-3006-4e30-bd4f-4104affaacb7") failed: Property not found. retry in 1.162130154s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_7ad61349-3006-4e30-bd4f-4104affaacb7
4922024-08-28T00:51:48.202ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_4dfcfd32-ab45-42c7-9c2f-4d836b875b6a") failed: Property not found. retry in 877.587445ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_4dfcfd32-ab45-42c7-9c2f-4d836b875b6a
4932024-08-28T00:51:48.992ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_09ed9686-d85a-429a-baae-bf2b96abe530") failed: Property not found. retry in 558.445134ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_09ed9686-d85a-429a-baae-bf2b96abe530
4942024-08-28T00:51:49.088ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_4dfcfd32-ab45-42c7-9c2f-4d836b875b6a") failed: Property not found. retry in 791.750299ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_4dfcfd32-ab45-42c7-9c2f-4d836b875b6a
4952024-08-28T00:51:49.320ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_7ad61349-3006-4e30-bd4f-4104affaacb7") failed: Property not found. retry in 982.534152ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_7ad61349-3006-4e30-bd4f-4104affaacb7
4962024-08-28T00:51:49.559ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_09ed9686-d85a-429a-baae-bf2b96abe530") failed: Property not found. retry in 1.412054965s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_09ed9686-d85a-429a-baae-bf2b96abe530
4972024-08-28T00:51:49.888ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_4dfcfd32-ab45-42c7-9c2f-4d836b875b6a") failed: Property not found. retry in 1.021375966s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_4dfcfd32-ab45-42c7-9c2f-4d836b875b6a
4982024-08-28T00:51:50.052ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::2 exists - OK file = sled-agent/src/services.rs:4206
4992024-08-28T00:51:50.053ZINFOSledAgent (ServiceManager): Ensuring there is a default route file = sled-agent/src/services.rs:4216 gateway = fd00:1122:3344:101::1
5002024-08-28T00:51:50.058ZINFOSledAgent (ServiceManager): configuring dendrite service file = sled-agent/src/services.rs:4295
5012024-08-28T00:51:50.177ZINFOSledAgent (ServiceManager): refreshed dendrite service with new configuration file = sled-agent/src/services.rs:4343
5022024-08-28T00:51:50.177ZINFOSledAgent (ServiceManager): configuring lldp service file = sled-agent/src/services.rs:4370
5032024-08-28T00:51:50.215ZINFOSledAgent (ServiceManager): refreshed lldpd service with new configuration file = sled-agent/src/services.rs:4383
5042024-08-28T00:51:50.215ZINFOSledAgent (ServiceManager): configuring MGS service file = sled-agent/src/services.rs:4247
5052024-08-28T00:51:50.266ZINFOSledAgent (ServiceManager): refreshed MGS service with new configuration file = sled-agent/src/services.rs:4289
5062024-08-28T00:51:50.266ZINFOSledAgent (ServiceManager): configuring wicketd file = sled-agent/src/services.rs:4350 rack_subnet = fd00:1122:3344::
5072024-08-28T00:51:50.285ZINFOSledAgent (ServiceManager): refreshed wicketd service with new configuration file = sled-agent/src/services.rs:4361
5082024-08-28T00:51:50.285ZINFOSledAgent (ServiceManager): configuring mgd service file = sled-agent/src/services.rs:4403
5092024-08-28T00:51:50.315ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_7ad61349-3006-4e30-bd4f-4104affaacb7") failed: Property not found. retry in 1.446753452s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_7ad61349-3006-4e30-bd4f-4104affaacb7
5102024-08-28T00:51:50.373ZINFOSledAgent (ServiceManager): refreshed mgd service with new configuration file = sled-agent/src/services.rs:4435
5112024-08-28T00:51:50.373ZINFOSledAgent (ServiceManager): configuring mg-ddm service file = sled-agent/src/services.rs:4441
5122024-08-28T00:51:50.486ZINFOSledAgent (ServiceManager): refreshed mg-ddm service with new configuration file = sled-agent/src/services.rs:4482
5132024-08-28T00:51:50.486ZINFOSledAgent (ServiceManager): Determining physical location of our switch zone at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:366
5142024-08-28T00:51:50.519ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/45733be1-aef6-43c8-be71-719893a1feda): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/nexus.rs:571 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
5152024-08-28T00:51:50.519ZWARNSledAgent (ServiceManager): Failed to get switch ID from MGS (retrying in 69.836498ms) error = Communication Error: error sending request for url (http://[fd00:1122:3344:101::2]:12225/local/switch-id): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/bootstrap/early_networking.rs:385
5162024-08-28T00:51:50.590ZWARNSledAgent (ServiceManager): Failed to get switch ID from MGS (retrying in 105.182534ms) error = Communication Error: error sending request for url (http://[fd00:1122:3344:101::2]:12225/local/switch-id): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/bootstrap/early_networking.rs:385
5172024-08-28T00:51:50.698ZINFOSledAgent (ServiceManager): Initializing 1 Uplinks on Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:414
5182024-08-28T00:51:50.717ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::2]:60702
5192024-08-28T00:51:50.717ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 54 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::2]:60702 req_id = 2fb4f49e-2351-495c-8091-55da38c14866 response_code = 200 uri = /sled-identifiers
5202024-08-28T00:51:50.736ZINFOSledAgent (ServiceManager): Building Port Configuration file = sled-agent/src/bootstrap/early_networking.rs:676
5212024-08-28T00:51:50.736ZINFOSledAgent (ServiceManager): Checking dendrite uptime file = sled-agent/src/bootstrap/early_networking.rs:714
5222024-08-28T00:51:50.736ZINFOSledAgent (ServiceManager): Dendrite online file = sled-agent/src/bootstrap/early_networking.rs:717 uptime = 0
5232024-08-28T00:51:50.736ZINFOSledAgent (ServiceManager): Configuring default uplink on switch config = PortSettings {\n links: {\n "0": LinkSettings {\n addrs: [\n 10.151.1.174,\n ],\n params: LinkCreate {\n autoneg: false,\n fec: None,\n kr: false,\n lane: Some(\n LinkId(\n 0,\n ),\n ),\n speed: Speed40G,\n },\n },\n },\n} file = sled-agent/src/bootstrap/early_networking.rs:436
5242024-08-28T00:51:50.773ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
5252024-08-28T00:51:50.825ZINFOSledAgent (StorageManager): Received DetectedRawDisksChanged { raw_disks: {}, tx: ... } file = sled-storage/src/manager.rs:406
5262024-08-28T00:51:50.825ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
5272024-08-28T00:51:50.825ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
5282024-08-28T00:51:50.825ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
5292024-08-28T00:51:50.825ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
5302024-08-28T00:51:50.825ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
5312024-08-28T00:51:50.825ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
5322024-08-28T00:51:50.825ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
5332024-08-28T00:51:50.825ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
5342024-08-28T00:51:50.825ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
5352024-08-28T00:51:50.825ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
5362024-08-28T00:51:50.825ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
5372024-08-28T00:51:50.825ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
5382024-08-28T00:51:50.825ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
5392024-08-28T00:51:50.825ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
5402024-08-28T00:51:50.825ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
5412024-08-28T00:51:50.825ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
5422024-08-28T00:51:50.825ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
5432024-08-28T00:51:50.825ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
5442024-08-28T00:51:50.825ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
5452024-08-28T00:51:50.825ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
5462024-08-28T00:51:50.825ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
5472024-08-28T00:51:50.825ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
5482024-08-28T00:51:50.917ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_4dfcfd32-ab45-42c7-9c2f-4d836b875b6a") failed: Property not found. retry in 994.284816ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_4dfcfd32-ab45-42c7-9c2f-4d836b875b6a
5492024-08-28T00:51:50.980ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_09ed9686-d85a-429a-baae-bf2b96abe530") failed: Property not found. retry in 1.284372716s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_09ed9686-d85a-429a-baae-bf2b96abe530
5502024-08-28T00:51:52.284ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_internal_dns_4dfcfd32-ab45-42c7-9c2f-4d836b875b6a
5512024-08-28T00:51:52.284ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_internal_dns_7ad61349-3006-4e30-bd4f-4104affaacb7
5522024-08-28T00:51:52.284ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_internal_dns_09ed9686-d85a-429a-baae-bf2b96abe530
5532024-08-28T00:51:52.284ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/1d8f47b2-ceb5-4768-b5e0-de73f9c6288a/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
5542024-08-28T00:51:52.285ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/2701509e-5560-4f63-bb12-b820fcc5e4cb/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
5552024-08-28T00:51:52.285ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8306720 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:50918 req_id = 5c2698da-3c4e-4af0-b320-1fc042b189f3 response_code = 204 uri = /omicron-zones
5562024-08-28T00:51:52.287ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:1::1]:5353 file = sled-agent/src/rack_setup/service.rs:555
5572024-08-28T00:51:52.333ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:2::1]:5353 file = sled-agent/src/rack_setup/service.rs:555
5582024-08-28T00:51:52.377ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:3::1]:5353 file = sled-agent/src/rack_setup/service.rs:555
5592024-08-28T00:51:52.424ZINFOSledAgent (RSS): Configured all DNS servers file = sled-agent/src/rack_setup/service.rs:591
5602024-08-28T00:51:52.424ZINFOSledAgent (RSS): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
5612024-08-28T00:51:52.458ZINFOSledAgent (RSS): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
5622024-08-28T00:51:52.459ZINFOSledAgent (RSS): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
5632024-08-28T00:51:52.459ZINFOSledAgent (RSS): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
5642024-08-28T00:51:52.489ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(3), zones: [OmicronZoneConfig { id: 7ad61349-3006-4e30-bd4f-4104affaacb7, underlay_address: fd00:1122:3344:1::1, filesystem_pool: Some(ZpoolName { id: 2deff32e-1106-452e-a0e4-7f5633a4a1c4 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 2deff32e-1106-452e-a0e4-7f5633a4a1c4 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 4dfcfd32-ab45-42c7-9c2f-4d836b875b6a, underlay_address: fd00:1122:3344:2::1, filesystem_pool: Some(ZpoolName { id: 903e5bdb-4fa3-466b-9cc6-cb6c1205c20b (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 903e5bdb-4fa3-466b-9cc6-cb6c1205c20b (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 09ed9686-d85a-429a-baae-bf2b96abe530, underlay_address: fd00:1122:3344:3::1, filesystem_pool: Some(ZpoolName { id: 8c0884cf-f16d-4d49-a677-c20655479168 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8c0884cf-f16d-4d49-a677-c20655479168 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 3bb25b5c-fb2f-4377-9789-fc9d2dfa852e, underlay_address: fd00:1122:3344:101::1b, filesystem_pool: Some(ZpoolName { id: 903e5bdb-4fa3-466b-9cc6-cb6c1205c20b (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 82978440-7287-48f8-b798-a45045d501a1, kind: Service { id: 3bb25b5c-fb2f-4377-9789-fc9d2dfa852e }, name: Name("ntp-3bb25b5c-fb2f-4377-9789-fc9d2dfa852e"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 221, 51])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } } }] } file = sled-agent/src/rack_setup/service.rs:438 sled_address = [fd00:1122:3344:101::1]:12345
5652024-08-28T00:51:52.489ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:60056
5662024-08-28T00:51:52.489ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 7ad61349-3006-4e30-bd4f-4104affaacb7, dataset_name: DatasetName { pool_name: ZpoolName { id: 2deff32e-1106-452e-a0e4-7f5633a4a1c4 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:406
5672024-08-28T00:51:52.489ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 7ad61349-3006-4e30-bd4f-4104affaacb7, dataset_name: DatasetName { pool_name: ZpoolName { id: 2deff32e-1106-452e-a0e4-7f5633a4a1c4 (zpool), kind: External }, kind: InternalDns }, responder: ... }
5682024-08-28T00:51:52.507ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 4dfcfd32-ab45-42c7-9c2f-4d836b875b6a, dataset_name: DatasetName { pool_name: ZpoolName { id: 903e5bdb-4fa3-466b-9cc6-cb6c1205c20b (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:406
5692024-08-28T00:51:52.507ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 4dfcfd32-ab45-42c7-9c2f-4d836b875b6a, dataset_name: DatasetName { pool_name: ZpoolName { id: 903e5bdb-4fa3-466b-9cc6-cb6c1205c20b (zpool), kind: External }, kind: InternalDns }, responder: ... }
5702024-08-28T00:51:52.523ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 09ed9686-d85a-429a-baae-bf2b96abe530, dataset_name: DatasetName { pool_name: ZpoolName { id: 8c0884cf-f16d-4d49-a677-c20655479168 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:406
5712024-08-28T00:51:52.523ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 09ed9686-d85a-429a-baae-bf2b96abe530, dataset_name: DatasetName { pool_name: ZpoolName { id: 8c0884cf-f16d-4d49-a677-c20655479168 (zpool), kind: External }, kind: InternalDns }, responder: ... }
5722024-08-28T00:51:52.538ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
5732024-08-28T00:51:52.539ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/1d8f47b2-ceb5-4768-b5e0-de73f9c6288a/config/omicron-zones.json file = common/src/ledger.rs:177
5742024-08-28T00:51:52.539ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/2701509e-5560-4f63-bb12-b820fcc5e4cb/config/omicron-zones.json file = common/src/ledger.rs:177
5752024-08-28T00:51:52.539ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
5762024-08-28T00:51:52.549ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
5772024-08-28T00:51:52.583ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
5782024-08-28T00:51:52.584ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
5792024-08-28T00:51:52.584ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
5802024-08-28T00:51:52.615ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte0", ip: 172.30.3.5, mac: MacAddr6([168, 64, 37, 255, 221, 51]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.3.1 } }) }
5812024-08-28T00:51:52.615ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1235 zone_type = boundary_ntp
5822024-08-28T00:51:52.616ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.1.169 file = clients/dpd-client/src/lib.rs:83
5832024-08-28T00:51:52.616ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.1.169 file = clients/dpd-client/src/lib.rs:90
5842024-08-28T00:51:52.616ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.1.169 file = clients/dpd-client/src/lib.rs:118
5852024-08-28T00:51:52.616ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
5862024-08-28T00:51:52.639ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_ntp_3bb25b5c-fb2f-4377-9789-fc9d2dfa852e file = illumos-utils/src/zone.rs:316
5872024-08-28T00:51:52.657ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_ntp_3bb25b5c-fb2f-4377-9789-fc9d2dfa852e file = illumos-utils/src/zone.rs:354
5882024-08-28T00:51:53.286ZINFOSledAgent (ServiceManager): Profile for oxz_ntp_3bb25b5c-fb2f-4377-9789-fc9d2dfa852e: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService4'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::1b'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/chrony-setup"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="allow" value='fd00:1122:3344:100::/56'/> <propval type="boolean" name="boundary" value='true'/> <propval type="astring" name="boundary_pool" value='boundary-ntp.control-plane.oxide.internal'/> <propval type="astring" name="server" value='0.pool.ntp.org'/> </property_group> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="1.1.1.1" /> <value_node value="9.9.9.9" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/ntp"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte0'/> <propval type="astring" name="gateway" value='172.30.3.1'/> <propval type="astring" name="ip" value='172.30.3.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
5892024-08-28T00:51:53.286ZINFOSledAgent (ServiceManager): Booting oxz_ntp_3bb25b5c-fb2f-4377-9789-fc9d2dfa852e zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_ntp_3bb25b5c-fb2f-4377-9789-fc9d2dfa852e
5902024-08-28T00:51:53.409ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_3bb25b5c-fb2f-4377-9789-fc9d2dfa852e") failed: Property not found. retry in 38.00595ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_3bb25b5c-fb2f-4377-9789-fc9d2dfa852e
5912024-08-28T00:51:53.454ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_3bb25b5c-fb2f-4377-9789-fc9d2dfa852e") failed: Property not found. retry in 98.04476ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_3bb25b5c-fb2f-4377-9789-fc9d2dfa852e
5922024-08-28T00:51:53.582ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_3bb25b5c-fb2f-4377-9789-fc9d2dfa852e") failed: Property not found. retry in 144.849154ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_3bb25b5c-fb2f-4377-9789-fc9d2dfa852e
5932024-08-28T00:51:53.739ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_3bb25b5c-fb2f-4377-9789-fc9d2dfa852e") failed: Property not found. retry in 385.426032ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_3bb25b5c-fb2f-4377-9789-fc9d2dfa852e
5942024-08-28T00:51:54.136ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_3bb25b5c-fb2f-4377-9789-fc9d2dfa852e") failed: Property not found. retry in 548.193241ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_3bb25b5c-fb2f-4377-9789-fc9d2dfa852e
5952024-08-28T00:51:54.694ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_3bb25b5c-fb2f-4377-9789-fc9d2dfa852e") failed: Property not found. retry in 513.128736ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_3bb25b5c-fb2f-4377-9789-fc9d2dfa852e
5962024-08-28T00:51:55.217ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_3bb25b5c-fb2f-4377-9789-fc9d2dfa852e") failed: Property not found. retry in 1.173398153s file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_3bb25b5c-fb2f-4377-9789-fc9d2dfa852e
5972024-08-28T00:51:56.399ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_3bb25b5c-fb2f-4377-9789-fc9d2dfa852e") failed: Property not found. retry in 574.727348ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_3bb25b5c-fb2f-4377-9789-fc9d2dfa852e
5982024-08-28T00:51:56.982ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_3bb25b5c-fb2f-4377-9789-fc9d2dfa852e") failed: Property not found. retry in 971.703109ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_3bb25b5c-fb2f-4377-9789-fc9d2dfa852e
5992024-08-28T00:51:57.964ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_3bb25b5c-fb2f-4377-9789-fc9d2dfa852e") failed: Property not found. retry in 599.514283ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_3bb25b5c-fb2f-4377-9789-fc9d2dfa852e
6002024-08-28T00:51:58.584ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_ntp_3bb25b5c-fb2f-4377-9789-fc9d2dfa852e
6012024-08-28T00:51:58.584ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/1d8f47b2-ceb5-4768-b5e0-de73f9c6288a/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
6022024-08-28T00:51:58.585ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/2701509e-5560-4f63-bb12-b820fcc5e4cb/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
6032024-08-28T00:51:58.585ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 6096284 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:60056 req_id = e4e8d21d-a9ca-41d9-8354-c00af9e5570a response_code = 204 uri = /omicron-zones
6042024-08-28T00:51:58.587ZINFOSledAgent (RSS): Waiting for rack time synchronization file = sled-agent/src/rack_setup/service.rs:632
6052024-08-28T00:51:58.618ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6062024-08-28T00:51:58.618ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:54119
6072024-08-28T00:51:58.627ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_3bb25b5c-fb2f-4377-9789-fc9d2dfa852e': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3847
6082024-08-28T00:51:58.627ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:902 latency_us = 8969 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:54119 req_id = d98a1c4f-6818-44ee-8315-b184745a893d response_code = 500 uri = /timesync
6092024-08-28T00:51:58.630ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6102024-08-28T00:51:58.738ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6112024-08-28T00:51:58.738ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:46451
6122024-08-28T00:52:02.746ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_3bb25b5c-fb2f-4377-9789-fc9d2dfa852e': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3847
6132024-08-28T00:52:02.746ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:902 latency_us = 4008140 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:46451 req_id = c4dd7748-ae9b-4540-8d95-55c93d016aea response_code = 500 uri = /timesync
6142024-08-28T00:52:02.750ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6152024-08-28T00:52:02.956ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6162024-08-28T00:52:02.956ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:54148
6172024-08-28T00:52:02.964ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_3bb25b5c-fb2f-4377-9789-fc9d2dfa852e': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3847
6182024-08-28T00:52:02.964ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:902 latency_us = 8681 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:54148 req_id = 02d920af-cf67-4b8c-be82-6b464724d03f response_code = 500 uri = /timesync
6192024-08-28T00:52:02.967ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6202024-08-28T00:52:03.208ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6212024-08-28T00:52:03.208ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:64435
6222024-08-28T00:52:03.216ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_3bb25b5c-fb2f-4377-9789-fc9d2dfa852e': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3847
6232024-08-28T00:52:03.216ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:902 latency_us = 7817 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:64435 req_id = 072b1b4e-6675-4e02-b959-53322391b494 response_code = 500 uri = /timesync
6242024-08-28T00:52:03.218ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6252024-08-28T00:52:03.396ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6262024-08-28T00:52:03.396ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:47589
6272024-08-28T00:52:07.890ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 4008973 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:47589 req_id = 852a90f3-9248-477c-92af-0609000a93d2 response_code = 200 uri = /timesync
6282024-08-28T00:52:07.892ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: -0.379773676 } file = sled-agent/src/rack_setup/service.rs:640
6292024-08-28T00:52:07.892ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6302024-08-28T00:52:08.088ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6312024-08-28T00:52:08.088ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:61897
6322024-08-28T00:52:08.097ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8531 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:61897 req_id = 50e2d565-e4ab-4fec-bcdc-a55547828f09 response_code = 200 uri = /timesync
6332024-08-28T00:52:08.099ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: -0.379210383 } file = sled-agent/src/rack_setup/service.rs:640
6342024-08-28T00:52:08.099ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6352024-08-28T00:52:08.277ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6362024-08-28T00:52:08.277ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:46102
6372024-08-28T00:52:08.286ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8856 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:46102 req_id = d2e6ff75-5237-404c-8087-155b28546eec response_code = 200 uri = /timesync
6382024-08-28T00:52:08.289ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: -0.378695995 } file = sled-agent/src/rack_setup/service.rs:640
6392024-08-28T00:52:08.289ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6402024-08-28T00:52:08.741ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6412024-08-28T00:52:08.741ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:57520
6422024-08-28T00:52:08.749ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8030 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:57520 req_id = e52b5370-f751-4e22-af32-317c396b96f3 response_code = 200 uri = /timesync
6432024-08-28T00:52:08.751ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2616771612, ip_addr: 155.248.196.28, stratum: 3, ref_time: 1724806327.9971592, correction: -0.378211319 } file = sled-agent/src/rack_setup/service.rs:640
6442024-08-28T00:52:08.752ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6452024-08-28T00:52:09.041ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6462024-08-28T00:52:09.042ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:38808
6472024-08-28T00:52:09.051ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 9266 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:38808 req_id = 99ef321d-4d71-4bb5-af33-a5d5fe4deef4 response_code = 200 uri = /timesync
6482024-08-28T00:52:09.053ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1724806326.7825043, correction: -0.377391875 } file = sled-agent/src/rack_setup/service.rs:640
6492024-08-28T00:52:09.053ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6502024-08-28T00:52:09.392ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6512024-08-28T00:52:09.392ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:53508
6522024-08-28T00:52:09.400ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 7822 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:53508 req_id = 7cf29fe6-ef3b-4aad-8154-da21c1084250 response_code = 200 uri = /timesync
6532024-08-28T00:52:09.403ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1724806326.7825043, correction: -0.376442164 } file = sled-agent/src/rack_setup/service.rs:640
6542024-08-28T00:52:09.403ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6552024-08-28T00:52:10.055ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6562024-08-28T00:52:10.055ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:61766
6572024-08-28T00:52:10.063ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8158 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:61766 req_id = fcaad9b7-a9ca-43cd-a877-e6bf524e7522 response_code = 200 uri = /timesync
6582024-08-28T00:52:10.065ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1724806326.7825043, correction: -0.374782801 } file = sled-agent/src/rack_setup/service.rs:640
6592024-08-28T00:52:10.065ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6602024-08-28T00:52:10.466ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6612024-08-28T00:52:10.466ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:38682
6622024-08-28T00:52:10.473ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 7595 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:38682 req_id = 5cf94e2b-0e56-4b16-b6aa-fbef7c71a6bf response_code = 200 uri = /timesync
6632024-08-28T00:52:10.477ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1724806326.7825043, correction: -0.373668253 } file = sled-agent/src/rack_setup/service.rs:640
6642024-08-28T00:52:10.477ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6652024-08-28T00:52:11.307ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6662024-08-28T00:52:11.307ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:44823
6672024-08-28T00:52:11.315ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8649 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:44823 req_id = 6967a47f-e3a3-4a61-9c63-2d19449a5058 response_code = 200 uri = /timesync
6682024-08-28T00:52:11.318ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1724806326.7825043, correction: -0.373032898 } file = sled-agent/src/rack_setup/service.rs:640
6692024-08-28T00:52:11.318ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6702024-08-28T00:52:12.161ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6712024-08-28T00:52:12.161ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:59410
6722024-08-28T00:52:12.170ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 9451 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:59410 req_id = 8152fc47-9bbd-4fba-bf74-669fd4e330bc response_code = 200 uri = /timesync
6732024-08-28T00:52:12.174ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2616771612, ip_addr: 155.248.196.28, stratum: 3, ref_time: 1724806330.2783256, correction: -0.370756119 } file = sled-agent/src/rack_setup/service.rs:640
6742024-08-28T00:52:12.174ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6752024-08-28T00:52:12.987ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6762024-08-28T00:52:12.987ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:55678
6772024-08-28T00:52:12.996ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8863 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:55678 req_id = 532d5f1a-74dd-4de3-bcba-7b4b89170cae response_code = 200 uri = /timesync
6782024-08-28T00:52:12.999ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2616771612, ip_addr: 155.248.196.28, stratum: 3, ref_time: 1724806331.669433, correction: -0.368195593 } file = sled-agent/src/rack_setup/service.rs:640
6792024-08-28T00:52:12.999ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6802024-08-28T00:52:13.730ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6812024-08-28T00:52:13.730ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:42667
6822024-08-28T00:52:13.738ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8461 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:42667 req_id = fec68ead-fbc0-4be7-924d-177ff1f05c22 response_code = 200 uri = /timesync
6832024-08-28T00:52:13.741ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2616771612, ip_addr: 155.248.196.28, stratum: 3, ref_time: 1724806332.7627618, correction: -0.365877718 } file = sled-agent/src/rack_setup/service.rs:640
6842024-08-28T00:52:13.741ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6852024-08-28T00:52:15.024ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6862024-08-28T00:52:15.025ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:63593
6872024-08-28T00:52:15.034ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 9576 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:63593 req_id = ebc8be79-affb-463c-927f-d100913e2084 response_code = 200 uri = /timesync
6882024-08-28T00:52:15.038ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2616771612, ip_addr: 155.248.196.28, stratum: 3, ref_time: 1724806334.0621784, correction: -0.360762626 } file = sled-agent/src/rack_setup/service.rs:640
6892024-08-28T00:52:15.038ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6902024-08-28T00:52:17.726ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6912024-08-28T00:52:17.726ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:41995
6922024-08-28T00:52:17.736ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 9469 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:41995 req_id = 30d615e8-e541-4518-a5e3-1ecc01e3f4a2 response_code = 200 uri = /timesync
6932024-08-28T00:52:17.739ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2616771612, ip_addr: 155.248.196.28, stratum: 3, ref_time: 1724806336.6798468, correction: -0.353381783 } file = sled-agent/src/rack_setup/service.rs:640
6942024-08-28T00:52:17.739ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6952024-08-28T00:52:20.078ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6962024-08-28T00:52:20.078ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:52160
6972024-08-28T00:52:20.088ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 9334 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:52160 req_id = 0e7605b1-2f1b-46a4-8e72-8d16975eae78 response_code = 200 uri = /timesync
6982024-08-28T00:52:20.092ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2616771612, ip_addr: 155.248.196.28, stratum: 3, ref_time: 1724806338.9374597, correction: -0.34701246 } file = sled-agent/src/rack_setup/service.rs:640
6992024-08-28T00:52:20.092ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
7002024-08-28T00:52:20.709ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:61200
7012024-08-28T00:52:20.709ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 58 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:61200 req_id = ea056f2c-6841-46ab-8b09-6935d0b0756e response_code = 200 uri = /baseboard
7022024-08-28T00:52:21.775ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
7032024-08-28T00:52:21.775ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:43153
7042024-08-28T00:52:21.783ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8701 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:43153 req_id = ce036adf-8d9f-4be6-b14d-49be04bf5ebd response_code = 200 uri = /timesync
7052024-08-28T00:52:21.787ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2616771612, ip_addr: 155.248.196.28, stratum: 3, ref_time: 1724806340.6356091, correction: -0.342390448 } file = sled-agent/src/rack_setup/service.rs:640
7062024-08-28T00:52:21.787ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
7072024-08-28T00:52:23.168ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 275.410202ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
7082024-08-28T00:52:26.496ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
7092024-08-28T00:52:26.496ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:36281
7102024-08-28T00:52:26.505ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8859 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:36281 req_id = a4b50973-1b46-4007-a7c0-dccfe3f0d0c0 response_code = 200 uri = /timesync
7112024-08-28T00:52:26.507ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2616771612, ip_addr: 155.248.196.28, stratum: 3, ref_time: 1724806345.2891445, correction: -0.329571009 } file = sled-agent/src/rack_setup/service.rs:640
7122024-08-28T00:52:26.507ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
7132024-08-28T00:52:29.299ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
7142024-08-28T00:52:29.299ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:37359
7152024-08-28T00:52:29.308ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8649 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:37359 req_id = d4a23aef-21d8-4eaa-80c3-a6fb96350f1c response_code = 200 uri = /timesync
7162024-08-28T00:52:29.312ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2616771612, ip_addr: 155.248.196.28, stratum: 3, ref_time: 1724806347.4616978, correction: -0.321962655 } file = sled-agent/src/rack_setup/service.rs:640
7172024-08-28T00:52:29.312ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
7182024-08-28T00:52:35.254ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
7192024-08-28T00:52:35.254ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:38312
7202024-08-28T00:52:35.263ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8859 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:38312 req_id = e12a589c-10dd-49b9-8fa1-cda4e8786944 response_code = 200 uri = /timesync
7212024-08-28T00:52:35.266ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2616771612, ip_addr: 155.248.196.28, stratum: 3, ref_time: 1724806354.3839843, correction: -0.305808097 } file = sled-agent/src/rack_setup/service.rs:640
7222024-08-28T00:52:35.266ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
7232024-08-28T00:52:38.436ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 748.668035ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
7242024-08-28T00:52:39.895ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
7252024-08-28T00:52:39.895ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:55679
7262024-08-28T00:52:39.904ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8965 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:55679 req_id = 832fcf6c-0c00-4c7a-9797-2bf245479a33 response_code = 200 uri = /timesync
7272024-08-28T00:52:39.908ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2616771612, ip_addr: 155.248.196.28, stratum: 3, ref_time: 1724806355.991222, correction: -0.293214977 } file = sled-agent/src/rack_setup/service.rs:640
7282024-08-28T00:52:39.908ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
7292024-08-28T00:52:44.441ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
7302024-08-28T00:52:44.441ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:36868
7312024-08-28T00:52:44.450ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 9110 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:36868 req_id = c5a6549c-407b-4bc5-986b-02b1054a8103 response_code = 200 uri = /timesync
7322024-08-28T00:52:44.454ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2616771612, ip_addr: 155.248.196.28, stratum: 3, ref_time: 1724806360.4003203, correction: -0.280877143 } file = sled-agent/src/rack_setup/service.rs:640
7332024-08-28T00:52:44.454ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
7342024-08-28T00:52:50.183ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
7352024-08-28T00:52:50.184ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:42916
7362024-08-28T00:52:50.192ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 7855 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:42916 req_id = dd041df5-e610-49ef-a7f0-d7667e7ae6ea response_code = 200 uri = /timesync
7372024-08-28T00:52:50.196ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2616771612, ip_addr: 155.248.196.28, stratum: 3, ref_time: 1724806368.719141, correction: -0.26530388 } file = sled-agent/src/rack_setup/service.rs:640
7382024-08-28T00:52:50.196ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
7392024-08-28T00:52:50.664ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:50420
7402024-08-28T00:52:50.664ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 44 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:50420 req_id = 24344eaf-f121-4320-ae0a-d849264410a5 response_code = 200 uri = /baseboard
7412024-08-28T00:52:54.178ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 533.567297ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
7422024-08-28T00:52:59.805ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
7432024-08-28T00:52:59.806ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:35349
7442024-08-28T00:52:59.815ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 9251 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:35349 req_id = 327e818f-f4a0-4a3e-9b17-5881274a072f response_code = 200 uri = /timesync
7452024-08-28T00:52:59.817ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2616771612, ip_addr: 155.248.196.28, stratum: 3, ref_time: 1724806372.751208, correction: -0.239186421 } file = sled-agent/src/rack_setup/service.rs:640
7462024-08-28T00:52:59.817ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
7472024-08-28T00:53:09.705ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 2.838084082s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
7482024-08-28T00:53:12.284ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
7492024-08-28T00:53:12.284ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:40694
7502024-08-28T00:53:12.293ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 9245 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:40694 req_id = 3d1662b2-96f3-4a6a-baa1-a6b49d27ecf4 response_code = 200 uri = /timesync
7512024-08-28T00:53:12.297ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2616771612, ip_addr: 155.248.196.28, stratum: 3, ref_time: 1724806388.5859396, correction: -0.205324963 } file = sled-agent/src/rack_setup/service.rs:640
7522024-08-28T00:53:12.297ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
7532024-08-28T00:53:20.618ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:61973
7542024-08-28T00:53:20.618ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 41 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:61973 req_id = c0364e86-4ab1-4299-b67f-16d1b4a3348f response_code = 200 uri = /baseboard
7552024-08-28T00:53:27.530ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 5.417742606s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
7562024-08-28T00:53:27.741ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
7572024-08-28T00:53:27.741ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:40684
7582024-08-28T00:53:27.750ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 9333 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:40684 req_id = a2dbdb10-4aa5-49e4-abbc-4ca562142c15 response_code = 200 uri = /timesync
7592024-08-28T00:53:27.754ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2616771612, ip_addr: 155.248.196.28, stratum: 3, ref_time: 1724806404.9912636, correction: -0.16338335 } file = sled-agent/src/rack_setup/service.rs:640
7602024-08-28T00:53:27.754ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
7612024-08-28T00:53:46.460ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
7622024-08-28T00:53:46.461ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:40702
7632024-08-28T00:53:46.470ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 9187 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:40702 req_id = c78cff81-77b9-4c8c-a029-768e72429f58 response_code = 200 uri = /timesync
7642024-08-28T00:53:46.473ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2616771612, ip_addr: 155.248.196.28, stratum: 3, ref_time: 1724806420.7152622, correction: -0.11257413 } file = sled-agent/src/rack_setup/service.rs:640
7652024-08-28T00:53:46.473ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
7662024-08-28T00:53:47.929ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 11.140749075s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
7672024-08-28T00:53:50.569ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:58776
7682024-08-28T00:53:50.569ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 36 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:58776 req_id = 61a821c1-f37b-431c-b490-32cf5fa1311c response_code = 200 uri = /baseboard
7692024-08-28T00:54:04.939ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
7702024-08-28T00:54:04.939ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:46146
7712024-08-28T00:54:04.948ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 9227 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:46146 req_id = c262f300-2927-4d34-9224-0106d8002d7d response_code = 200 uri = /timesync
7722024-08-28T00:54:04.950ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2616771612, ip_addr: 155.248.196.28, stratum: 3, ref_time: 1724806436.852016, correction: -0.061942115 } file = sled-agent/src/rack_setup/service.rs:640
7732024-08-28T00:54:04.950ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
7742024-08-28T00:54:14.033ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 16.038302739s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
7752024-08-28T00:54:20.523ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:58708
7762024-08-28T00:54:20.523ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 47 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:58708 req_id = d93d2047-d7fd-46b5-be81-04fb5691efd5 response_code = 200 uri = /baseboard
7772024-08-28T00:54:22.958ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
7782024-08-28T00:54:22.958ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:57770
7792024-08-28T00:54:22.979ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 21415 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:57770 req_id = 154e9758-deb3-4523-8e34-e628b23c1580 response_code = 200 uri = /timesync
7802024-08-28T00:54:22.984ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: true, ref_id: 1097084582, ip_addr: 65.100.46.166, stratum: 2, ref_time: 1724806461.130667, correction: -0.012908353 } file = sled-agent/src/rack_setup/service.rs:640
7812024-08-28T00:54:22.985ZINFOSledAgent (RSS): Finished setting up Internal DNS and NTP file = sled-agent/src/rack_setup/service.rs:1266
7822024-08-28T00:54:23.015ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 7ad61349-3006-4e30-bd4f-4104affaacb7, underlay_address: fd00:1122:3344:1::1, filesystem_pool: Some(ZpoolName { id: 2deff32e-1106-452e-a0e4-7f5633a4a1c4 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 2deff32e-1106-452e-a0e4-7f5633a4a1c4 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 4dfcfd32-ab45-42c7-9c2f-4d836b875b6a, underlay_address: fd00:1122:3344:2::1, filesystem_pool: Some(ZpoolName { id: 903e5bdb-4fa3-466b-9cc6-cb6c1205c20b (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 903e5bdb-4fa3-466b-9cc6-cb6c1205c20b (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 09ed9686-d85a-429a-baae-bf2b96abe530, underlay_address: fd00:1122:3344:3::1, filesystem_pool: Some(ZpoolName { id: 8c0884cf-f16d-4d49-a677-c20655479168 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8c0884cf-f16d-4d49-a677-c20655479168 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 3bb25b5c-fb2f-4377-9789-fc9d2dfa852e, underlay_address: fd00:1122:3344:101::1b, filesystem_pool: Some(ZpoolName { id: 903e5bdb-4fa3-466b-9cc6-cb6c1205c20b (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 82978440-7287-48f8-b798-a45045d501a1, kind: Service { id: 3bb25b5c-fb2f-4377-9789-fc9d2dfa852e }, name: Name("ntp-3bb25b5c-fb2f-4377-9789-fc9d2dfa852e"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 221, 51])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 62d7b608-ad89-43a5-a5eb-30c5f510a264, underlay_address: fd00:1122:3344:101::3, filesystem_pool: Some(ZpoolName { id: 2deff32e-1106-452e-a0e4-7f5633a4a1c4 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 2deff32e-1106-452e-a0e4-7f5633a4a1c4 (zpool), kind: External } } } }, OmicronZoneConfig { id: c21bdc0c-57c3-4a54-aa95-85bc9627d5b9, underlay_address: fd00:1122:3344:101::4, filesystem_pool: Some(ZpoolName { id: 903e5bdb-4fa3-466b-9cc6-cb6c1205c20b (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 903e5bdb-4fa3-466b-9cc6-cb6c1205c20b (zpool), kind: External } } } }, OmicronZoneConfig { id: 2f3a388e-ad90-4757-82e1-21643e753681, underlay_address: fd00:1122:3344:101::5, filesystem_pool: Some(ZpoolName { id: 8c0884cf-f16d-4d49-a677-c20655479168 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8c0884cf-f16d-4d49-a677-c20655479168 (zpool), kind: External } } } }, OmicronZoneConfig { id: fdb1d73c-f437-4e37-b800-c198de5adf74, underlay_address: fd00:1122:3344:101::6, filesystem_pool: Some(ZpoolName { id: 39cc409d-9820-46b7-b64f-11f4d431b612 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 39cc409d-9820-46b7-b64f-11f4d431b612 (zpool), kind: External } } } }, OmicronZoneConfig { id: bab3ad8b-d8a3-4d57-816f-a33fdf2870fb, underlay_address: fd00:1122:3344:101::7, filesystem_pool: Some(ZpoolName { id: 753cff17-fbdf-48d1-960a-a1fe6d8c01ed (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 753cff17-fbdf-48d1-960a-a1fe6d8c01ed (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:438 sled_address = [fd00:1122:3344:101::1]:12345
7832024-08-28T00:54:23.015ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:38187
7842024-08-28T00:54:23.016ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 7ad61349-3006-4e30-bd4f-4104affaacb7, dataset_name: DatasetName { pool_name: ZpoolName { id: 2deff32e-1106-452e-a0e4-7f5633a4a1c4 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:406
7852024-08-28T00:54:23.016ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 7ad61349-3006-4e30-bd4f-4104affaacb7, dataset_name: DatasetName { pool_name: ZpoolName { id: 2deff32e-1106-452e-a0e4-7f5633a4a1c4 (zpool), kind: External }, kind: InternalDns }, responder: ... }
7862024-08-28T00:54:23.039ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 4dfcfd32-ab45-42c7-9c2f-4d836b875b6a, dataset_name: DatasetName { pool_name: ZpoolName { id: 903e5bdb-4fa3-466b-9cc6-cb6c1205c20b (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:406
7872024-08-28T00:54:23.039ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 4dfcfd32-ab45-42c7-9c2f-4d836b875b6a, dataset_name: DatasetName { pool_name: ZpoolName { id: 903e5bdb-4fa3-466b-9cc6-cb6c1205c20b (zpool), kind: External }, kind: InternalDns }, responder: ... }
7882024-08-28T00:54:23.059ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 09ed9686-d85a-429a-baae-bf2b96abe530, dataset_name: DatasetName { pool_name: ZpoolName { id: 8c0884cf-f16d-4d49-a677-c20655479168 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:406
7892024-08-28T00:54:23.059ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 09ed9686-d85a-429a-baae-bf2b96abe530, dataset_name: DatasetName { pool_name: ZpoolName { id: 8c0884cf-f16d-4d49-a677-c20655479168 (zpool), kind: External }, kind: InternalDns }, responder: ... }
7902024-08-28T00:54:23.079ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 62d7b608-ad89-43a5-a5eb-30c5f510a264, dataset_name: DatasetName { pool_name: ZpoolName { id: 2deff32e-1106-452e-a0e4-7f5633a4a1c4 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:406
7912024-08-28T00:54:23.079ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 62d7b608-ad89-43a5-a5eb-30c5f510a264, dataset_name: DatasetName { pool_name: ZpoolName { id: 2deff32e-1106-452e-a0e4-7f5633a4a1c4 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
7922024-08-28T00:54:23.133ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c21bdc0c-57c3-4a54-aa95-85bc9627d5b9, dataset_name: DatasetName { pool_name: ZpoolName { id: 903e5bdb-4fa3-466b-9cc6-cb6c1205c20b (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:406
7932024-08-28T00:54:23.133ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: c21bdc0c-57c3-4a54-aa95-85bc9627d5b9, dataset_name: DatasetName { pool_name: ZpoolName { id: 903e5bdb-4fa3-466b-9cc6-cb6c1205c20b (zpool), kind: External }, kind: CockroachDb }, responder: ... }
7942024-08-28T00:54:23.189ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2f3a388e-ad90-4757-82e1-21643e753681, dataset_name: DatasetName { pool_name: ZpoolName { id: 8c0884cf-f16d-4d49-a677-c20655479168 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:406
7952024-08-28T00:54:23.189ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 2f3a388e-ad90-4757-82e1-21643e753681, dataset_name: DatasetName { pool_name: ZpoolName { id: 8c0884cf-f16d-4d49-a677-c20655479168 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
7962024-08-28T00:54:23.241ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: fdb1d73c-f437-4e37-b800-c198de5adf74, dataset_name: DatasetName { pool_name: ZpoolName { id: 39cc409d-9820-46b7-b64f-11f4d431b612 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:406
7972024-08-28T00:54:23.241ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: fdb1d73c-f437-4e37-b800-c198de5adf74, dataset_name: DatasetName { pool_name: ZpoolName { id: 39cc409d-9820-46b7-b64f-11f4d431b612 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
7982024-08-28T00:54:23.293ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: bab3ad8b-d8a3-4d57-816f-a33fdf2870fb, dataset_name: DatasetName { pool_name: ZpoolName { id: 753cff17-fbdf-48d1-960a-a1fe6d8c01ed (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:406
7992024-08-28T00:54:23.293ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: bab3ad8b-d8a3-4d57-816f-a33fdf2870fb, dataset_name: DatasetName { pool_name: ZpoolName { id: 753cff17-fbdf-48d1-960a-a1fe6d8c01ed (zpool), kind: External }, kind: CockroachDb }, responder: ... }
8002024-08-28T00:54:23.344ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
8012024-08-28T00:54:23.344ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/1d8f47b2-ceb5-4768-b5e0-de73f9c6288a/config/omicron-zones.json file = common/src/ledger.rs:177
8022024-08-28T00:54:23.344ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/2701509e-5560-4f63-bb12-b820fcc5e4cb/config/omicron-zones.json file = common/src/ledger.rs:177
8032024-08-28T00:54:23.344ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
8042024-08-28T00:54:23.425ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
8052024-08-28T00:54:23.426ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
8062024-08-28T00:54:23.426ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
8072024-08-28T00:54:23.426ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
8082024-08-28T00:54:23.426ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
8092024-08-28T00:54:23.515ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681 file = illumos-utils/src/zone.rs:316
8102024-08-28T00:54:23.519ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74 file = illumos-utils/src/zone.rs:316
8112024-08-28T00:54:23.523ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9 file = illumos-utils/src/zone.rs:316
8122024-08-28T00:54:23.528ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb file = illumos-utils/src/zone.rs:316
8132024-08-28T00:54:23.532ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264 file = illumos-utils/src/zone.rs:316
8142024-08-28T00:54:23.537ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681 file = illumos-utils/src/zone.rs:354
8152024-08-28T00:54:23.541ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74 file = illumos-utils/src/zone.rs:354
8162024-08-28T00:54:23.545ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9 file = illumos-utils/src/zone.rs:354
8172024-08-28T00:54:23.550ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb file = illumos-utils/src/zone.rs:354
8182024-08-28T00:54:23.554ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264 file = illumos-utils/src/zone.rs:354
8192024-08-28T00:54:26.522ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService5'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::5]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='2f3a388e-ad90-4757-82e1-21643e753681'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::5]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::5]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:1::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8202024-08-28T00:54:26.522ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681
8212024-08-28T00:54:26.528ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService7'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::4'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::4]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='c21bdc0c-57c3-4a54-aa95-85bc9627d5b9'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::4]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::4]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:1::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8222024-08-28T00:54:26.528ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9
8232024-08-28T00:54:26.544ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService8'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::7'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::7]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='bab3ad8b-d8a3-4d57-816f-a33fdf2870fb'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::7]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::7]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:1::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8242024-08-28T00:54:26.544ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb
8252024-08-28T00:54:26.597ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService9'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::3'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::3]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='62d7b608-ad89-43a5-a5eb-30c5f510a264'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::3]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::3]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:1::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8262024-08-28T00:54:26.597ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264
8272024-08-28T00:54:26.619ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService6'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::6]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='fdb1d73c-f437-4e37-b800-c198de5adf74'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::6]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::6]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:1::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8282024-08-28T00:54:26.619ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74
8292024-08-28T00:54:26.669ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681") failed: Property not found. retry in 56.836864ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681
8302024-08-28T00:54:26.679ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9") failed: Property not found. retry in 25.307245ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9
8312024-08-28T00:54:26.686ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb") failed: Property not found. retry in 41.626104ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb
8322024-08-28T00:54:26.713ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9") failed: Property not found. retry in 129.909307ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9
8332024-08-28T00:54:26.736ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264") failed: Property not found. retry in 28.839996ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264
8342024-08-28T00:54:26.847ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681") failed: Property not found. retry in 142.632894ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681
8352024-08-28T00:54:27.150ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb") failed: Property not found. retry in 83.832435ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb
8362024-08-28T00:54:27.645ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681") failed: Property not found. retry in 173.169153ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681
8372024-08-28T00:54:27.692ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74") failed: Property not found. retry in 67.277816ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74
8382024-08-28T00:54:28.119ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9") failed: Property not found. retry in 293.008847ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9
8392024-08-28T00:54:28.203ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb") failed: Property not found. retry in 220.743715ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb
8402024-08-28T00:54:28.494ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264") failed: Property not found. retry in 86.995496ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264
8412024-08-28T00:54:28.738ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681") failed: Property not found. retry in 292.59925ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681
8422024-08-28T00:54:28.854ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74") failed: Property not found. retry in 128.579825ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74
8432024-08-28T00:54:28.932ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9") failed: Property not found. retry in 357.818257ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9
8442024-08-28T00:54:28.989ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb") failed: Property not found. retry in 449.994054ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb
8452024-08-28T00:54:29.076ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264") failed: Property not found. retry in 133.180448ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264
8462024-08-28T00:54:29.120ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681") failed: Property not found. retry in 807.210243ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681
8472024-08-28T00:54:29.161ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74") failed: Property not found. retry in 173.200222ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74
8482024-08-28T00:54:29.242ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264") failed: Property not found. retry in 206.557087ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264
8492024-08-28T00:54:29.300ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9") failed: Property not found. retry in 582.038613ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9
8502024-08-28T00:54:29.346ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74") failed: Property not found. retry in 290.981303ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74
8512024-08-28T00:54:29.452ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb") failed: Property not found. retry in 973.145933ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb
8522024-08-28T00:54:29.461ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264") failed: Property not found. retry in 1.166742788s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264
8532024-08-28T00:54:29.648ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74") failed: Property not found. retry in 789.577567ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74
8542024-08-28T00:54:29.893ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9") failed: Property not found. retry in 957.388849ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9
8552024-08-28T00:54:29.936ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681") failed: Property not found. retry in 1.494498653s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681
8562024-08-28T00:54:30.435ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb") failed: Property not found. retry in 1.469469086s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb
8572024-08-28T00:54:30.447ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74") failed: Property not found. retry in 1.03925918s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74
8582024-08-28T00:54:30.638ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264") failed: Property not found. retry in 891.414453ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264
8592024-08-28T00:54:30.861ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9") failed: Property not found. retry in 1.002037572s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9
8602024-08-28T00:54:31.441ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681") failed: Property not found. retry in 596.546263ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681
8612024-08-28T00:54:31.496ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74") failed: Property not found. retry in 531.270336ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74
8622024-08-28T00:54:31.539ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264") failed: Property not found. retry in 500.88876ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264
8632024-08-28T00:54:31.872ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9") failed: Property not found. retry in 1.380043737s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9
8642024-08-28T00:54:31.914ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb") failed: Property not found. retry in 515.096166ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb
8652024-08-28T00:54:32.037ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74") failed: Property not found. retry in 1.410062158s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74
8662024-08-28T00:54:32.046ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681") failed: Property not found. retry in 1.424246398s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681
8672024-08-28T00:54:32.053ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264") failed: Property not found. retry in 807.760113ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264
8682024-08-28T00:54:32.438ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb") failed: Property not found. retry in 1.047056217s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb
8692024-08-28T00:54:32.871ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264") failed: Property not found. retry in 691.907127ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264
8702024-08-28T00:54:33.262ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9") failed: Property not found. retry in 669.902217ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9
8712024-08-28T00:54:33.456ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74") failed: Property not found. retry in 1.436468692s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74
8722024-08-28T00:54:33.478ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681") failed: Property not found. retry in 1.285050044s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681
8732024-08-28T00:54:33.494ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb") failed: Property not found. retry in 835.506414ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb
8742024-08-28T00:54:33.572ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264") failed: Property not found. retry in 1.148602286s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264
8752024-08-28T00:54:33.941ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9") failed: Property not found. retry in 878.89051ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9
8762024-08-28T00:54:34.341ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb") failed: Property not found. retry in 1.137883729s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb
8772024-08-28T00:54:34.730ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264") failed: Property not found. retry in 1.022399851s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264
8782024-08-28T00:54:34.773ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681") failed: Property not found. retry in 929.021739ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681
8792024-08-28T00:54:34.829ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9") failed: Property not found. retry in 992.940744ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9
8802024-08-28T00:54:34.902ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74") failed: Property not found. retry in 1.14080003s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74
8812024-08-28T00:54:35.488ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb") failed: Property not found. retry in 618.034422ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb
8822024-08-28T00:54:35.711ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681") failed: Property not found. retry in 1.258660721s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681
8832024-08-28T00:54:35.762ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264") failed: Property not found. retry in 1.416612525s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264
8842024-08-28T00:54:35.831ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9") failed: Property not found. retry in 1.423197022s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9
8852024-08-28T00:54:36.052ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74") failed: Property not found. retry in 1.140746244s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74
8862024-08-28T00:54:36.115ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb") failed: Property not found. retry in 884.330962ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb
8872024-08-28T00:54:36.980ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681") failed: Property not found. retry in 1.106348967s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681
8882024-08-28T00:54:37.008ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb") failed: Property not found. retry in 1.401399965s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb
8892024-08-28T00:54:37.189ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264") failed: Property not found. retry in 1.052116685s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264
8902024-08-28T00:54:37.201ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74") failed: Property not found. retry in 836.588065ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74
8912024-08-28T00:54:37.263ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9") failed: Property not found. retry in 853.721409ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9
8922024-08-28T00:54:38.047ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74") failed: Property not found. retry in 524.583264ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74
8932024-08-28T00:54:38.095ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681") failed: Property not found. retry in 606.205026ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681
8942024-08-28T00:54:38.125ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9") failed: Property not found. retry in 1.035018842s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9
8952024-08-28T00:54:38.250ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264") failed: Property not found. retry in 1.066139727s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264
8962024-08-28T00:54:38.418ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb") failed: Property not found. retry in 1.352287938s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb
8972024-08-28T00:54:38.581ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74") failed: Property not found. retry in 811.087055ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74
8982024-08-28T00:54:38.710ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681") failed: Property not found. retry in 1.438361805s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681
8992024-08-28T00:54:39.171ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9") failed: Property not found. retry in 700.43686ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9
9002024-08-28T00:54:39.326ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264") failed: Property not found. retry in 941.225764ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264
9012024-08-28T00:54:39.400ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74") failed: Property not found. retry in 820.209203ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74
9022024-08-28T00:54:39.779ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb") failed: Property not found. retry in 910.601825ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb
9032024-08-28T00:54:39.881ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9") failed: Property not found. retry in 1.091752009s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9
9042024-08-28T00:54:40.157ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681") failed: Property not found. retry in 1.170763524s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681
9052024-08-28T00:54:40.230ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74") failed: Property not found. retry in 521.874682ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74
9062024-08-28T00:54:40.275ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264") failed: Property not found. retry in 1.205923998s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264
9072024-08-28T00:54:40.699ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb") failed: Property not found. retry in 636.703667ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb
9082024-08-28T00:54:40.761ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74") failed: Property not found. retry in 1.292201179s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74
9092024-08-28T00:54:40.982ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9") failed: Property not found. retry in 1.314088348s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9
9102024-08-28T00:54:41.338ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681") failed: Property not found. retry in 1.131547356s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681
9112024-08-28T00:54:41.345ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb") failed: Property not found. retry in 1.178277389s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb
9122024-08-28T00:54:41.500ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264") failed: Property not found. retry in 1.228074996s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264
9132024-08-28T00:54:42.064ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74") failed: Property not found. retry in 754.204933ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74
9142024-08-28T00:54:42.305ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9") failed: Property not found. retry in 794.546263ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9
9152024-08-28T00:54:42.480ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681") failed: Property not found. retry in 751.621027ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681
9162024-08-28T00:54:42.534ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb") failed: Property not found. retry in 571.418113ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb
9172024-08-28T00:54:42.740ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264") failed: Property not found. retry in 951.091936ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264
9182024-08-28T00:54:42.832ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74") failed: Property not found. retry in 1.385452739s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74
9192024-08-28T00:54:43.110ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9") failed: Property not found. retry in 650.288622ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9
9202024-08-28T00:54:43.119ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb") failed: Property not found. retry in 968.444102ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb
9212024-08-28T00:54:43.242ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681") failed: Property not found. retry in 539.114586ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681
9222024-08-28T00:54:43.701ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264") failed: Property not found. retry in 1.389363136s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264
9232024-08-28T00:54:43.770ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9") failed: Property not found. retry in 1.283419028s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9
9242024-08-28T00:54:43.789ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681") failed: Property not found. retry in 645.564605ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681
9252024-08-28T00:54:44.097ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb") failed: Property not found. retry in 607.333699ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb
9262024-08-28T00:54:44.226ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74") failed: Property not found. retry in 837.848987ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74
9272024-08-28T00:54:44.443ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681") failed: Property not found. retry in 739.959949ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681
9282024-08-28T00:54:44.714ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb") failed: Property not found. retry in 518.953125ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb
9292024-08-28T00:54:45.064ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9") failed: Property not found. retry in 1.302836896s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9
9302024-08-28T00:54:45.072ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74") failed: Property not found. retry in 1.264775681s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74
9312024-08-28T00:54:45.074ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 41.769908871s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
9322024-08-28T00:54:45.099ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264") failed: Property not found. retry in 1.243036033s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264
9332024-08-28T00:54:45.193ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681") failed: Property not found. retry in 872.001504ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681
9342024-08-28T00:54:45.242ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb") failed: Property not found. retry in 1.187476592s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb
9352024-08-28T00:54:46.075ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681") failed: Property not found. retry in 861.422074ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681
9362024-08-28T00:54:46.366ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264") failed: Property not found. retry in 682.188382ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264
9372024-08-28T00:54:47.073ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_cockroachdb_2f3a388e-ad90-4757-82e1-21643e753681
9382024-08-28T00:54:47.073ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_cockroachdb_fdb1d73c-f437-4e37-b800-c198de5adf74
9392024-08-28T00:54:47.073ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_cockroachdb_c21bdc0c-57c3-4a54-aa95-85bc9627d5b9
9402024-08-28T00:54:47.073ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_cockroachdb_bab3ad8b-d8a3-4d57-816f-a33fdf2870fb
9412024-08-28T00:54:47.073ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_cockroachdb_62d7b608-ad89-43a5-a5eb-30c5f510a264
9422024-08-28T00:54:47.073ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/1d8f47b2-ceb5-4768-b5e0-de73f9c6288a/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
9432024-08-28T00:54:47.073ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/2701509e-5560-4f63-bb12-b820fcc5e4cb/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
9442024-08-28T00:54:47.074ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 24070961 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:38187 req_id = 1fb0a775-ad74-48f2-a545-5b8e362ef61c response_code = 204 uri = /omicron-zones
9452024-08-28T00:54:47.109ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:35461
9462024-08-28T00:54:47.142ZINFOSledAgent (ServiceManager): Initializing CRDB Cluster - sending request to [fd00:1122:3344:101::5]:32221 file = sled-agent/src/services.rs:3681
9472024-08-28T00:54:52.720ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:43649
9482024-08-28T00:54:52.764ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 6277 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:43649 req_id = bebeb3f9-d42e-4087-bf7b-6e1e3dbfabc6 response_code = 200 uri = /baseboard
9492024-08-28T00:55:04.787ZINFOSledAgent (ServiceManager): Formatting CRDB file = sled-agent/src/services.rs:3699
9502024-08-28T00:55:11.455ZINFOSledAgent (ServiceManager): Formatting CRDB - Completed file = sled-agent/src/services.rs:3722
9512024-08-28T00:55:11.456ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 24346553 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:35461 req_id = f880a2ef-1c82-4cd0-98fb-146657f226da response_code = 204 uri = /cockroachdb
9522024-08-28T00:55:11.509ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(5), zones: [OmicronZoneConfig { id: 7ad61349-3006-4e30-bd4f-4104affaacb7, underlay_address: fd00:1122:3344:1::1, filesystem_pool: Some(ZpoolName { id: 2deff32e-1106-452e-a0e4-7f5633a4a1c4 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 2deff32e-1106-452e-a0e4-7f5633a4a1c4 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 4dfcfd32-ab45-42c7-9c2f-4d836b875b6a, underlay_address: fd00:1122:3344:2::1, filesystem_pool: Some(ZpoolName { id: 903e5bdb-4fa3-466b-9cc6-cb6c1205c20b (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 903e5bdb-4fa3-466b-9cc6-cb6c1205c20b (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 09ed9686-d85a-429a-baae-bf2b96abe530, underlay_address: fd00:1122:3344:3::1, filesystem_pool: Some(ZpoolName { id: 8c0884cf-f16d-4d49-a677-c20655479168 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8c0884cf-f16d-4d49-a677-c20655479168 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 3bb25b5c-fb2f-4377-9789-fc9d2dfa852e, underlay_address: fd00:1122:3344:101::1b, filesystem_pool: Some(ZpoolName { id: 903e5bdb-4fa3-466b-9cc6-cb6c1205c20b (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 82978440-7287-48f8-b798-a45045d501a1, kind: Service { id: 3bb25b5c-fb2f-4377-9789-fc9d2dfa852e }, name: Name("ntp-3bb25b5c-fb2f-4377-9789-fc9d2dfa852e"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 221, 51])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 62d7b608-ad89-43a5-a5eb-30c5f510a264, underlay_address: fd00:1122:3344:101::3, filesystem_pool: Some(ZpoolName { id: 2deff32e-1106-452e-a0e4-7f5633a4a1c4 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 2deff32e-1106-452e-a0e4-7f5633a4a1c4 (zpool), kind: External } } } }, OmicronZoneConfig { id: c21bdc0c-57c3-4a54-aa95-85bc9627d5b9, underlay_address: fd00:1122:3344:101::4, filesystem_pool: Some(ZpoolName { id: 903e5bdb-4fa3-466b-9cc6-cb6c1205c20b (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 903e5bdb-4fa3-466b-9cc6-cb6c1205c20b (zpool), kind: External } } } }, OmicronZoneConfig { id: 2f3a388e-ad90-4757-82e1-21643e753681, underlay_address: fd00:1122:3344:101::5, filesystem_pool: Some(ZpoolName { id: 8c0884cf-f16d-4d49-a677-c20655479168 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8c0884cf-f16d-4d49-a677-c20655479168 (zpool), kind: External } } } }, OmicronZoneConfig { id: fdb1d73c-f437-4e37-b800-c198de5adf74, underlay_address: fd00:1122:3344:101::6, filesystem_pool: Some(ZpoolName { id: 39cc409d-9820-46b7-b64f-11f4d431b612 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 39cc409d-9820-46b7-b64f-11f4d431b612 (zpool), kind: External } } } }, OmicronZoneConfig { id: bab3ad8b-d8a3-4d57-816f-a33fdf2870fb, underlay_address: fd00:1122:3344:101::7, filesystem_pool: Some(ZpoolName { id: 753cff17-fbdf-48d1-960a-a1fe6d8c01ed (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 753cff17-fbdf-48d1-960a-a1fe6d8c01ed (zpool), kind: External } } } }, OmicronZoneConfig { id: fa9e7556-b41e-4114-bf3f-e918820a9bcf, underlay_address: fd00:1122:3344:101::8, filesystem_pool: Some(ZpoolName { id: 2deff32e-1106-452e-a0e4-7f5633a4a1c4 (zpool), kind: External }), zone_type: ExternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 2deff32e-1106-452e-a0e4-7f5633a4a1c4 (zpool), kind: External } }, http_address: [fd00:1122:3344:101::8]:5353, dns_address: 10.151.1.164:53, nic: NetworkInterface { id: eafd744b-ffb5-445c-8366-bd33299374a6, kind: Service { id: fa9e7556-b41e-4114-bf3f-e918820a9bcf }, name: Name("external-dns-fa9e7556-b41e-4114-bf3f-e918820a9bcf"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 202, 132])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } } }, OmicronZoneConfig { id: 2cae9f85-8043-498f-af8e-a4bc9724ac55, underlay_address: fd00:1122:3344:101::9, filesystem_pool: Some(ZpoolName { id: 903e5bdb-4fa3-466b-9cc6-cb6c1205c20b (zpool), kind: External }), zone_type: ExternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 903e5bdb-4fa3-466b-9cc6-cb6c1205c20b (zpool), kind: External } }, http_address: [fd00:1122:3344:101::9]:5353, dns_address: 10.151.1.165:53, nic: NetworkInterface { id: 1eb980bf-46dc-45a7-aa4f-098bf58d6904, kind: Service { id: 2cae9f85-8043-498f-af8e-a4bc9724ac55 }, name: Name("external-dns-2cae9f85-8043-498f-af8e-a4bc9724ac55"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 212, 38])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } } }, OmicronZoneConfig { id: a5227819-155d-4a08-8e71-bd8a327ad739, underlay_address: fd00:1122:3344:101::a, filesystem_pool: Some(ZpoolName { id: 9b7eedae-40f6-4e96-bd9d-f89d715d8ad7 (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::a]:12221, external_ip: 10.151.1.166, nic: NetworkInterface { id: 707af25c-dc9c-42b2-9351-0dc3824f26c9, kind: Service { id: a5227819-155d-4a08-8e71-bd8a327ad739 }, name: Name("nexus-a5227819-155d-4a08-8e71-bd8a327ad739"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 220, 190])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: f840e071-026c-488d-b1c9-2c4d1ce3e74d, underlay_address: fd00:1122:3344:101::b, filesystem_pool: Some(ZpoolName { id: 8c0884cf-f16d-4d49-a677-c20655479168 (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::b]:12221, external_ip: 10.151.1.167, nic: NetworkInterface { id: 16870a00-56de-4335-bbe4-e10f8544f208, kind: Service { id: f840e071-026c-488d-b1c9-2c4d1ce3e74d }, name: Name("nexus-f840e071-026c-488d-b1c9-2c4d1ce3e74d"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 209, 34])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: 4ee19452-709a-4abd-a238-7af80f61d166, underlay_address: fd00:1122:3344:101::c, filesystem_pool: Some(ZpoolName { id: 903e5bdb-4fa3-466b-9cc6-cb6c1205c20b (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::c]:12221, external_ip: 10.151.1.168, nic: NetworkInterface { id: a6310629-11e4-4b6d-b328-1d8767a4ab69, kind: Service { id: 4ee19452-709a-4abd-a238-7af80f61d166 }, name: Name("nexus-4ee19452-709a-4abd-a238-7af80f61d166"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 191, 68])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: 4fe6d00e-187b-4dd4-9562-e20645ddec26, underlay_address: fd00:1122:3344:101::d, filesystem_pool: Some(ZpoolName { id: 9b7eedae-40f6-4e96-bd9d-f89d715d8ad7 (zpool), kind: External }), zone_type: Oximeter { address: [fd00:1122:3344:101::d]:12223 } }, OmicronZoneConfig { id: b5e20b06-936c-44f5-a47b-458ef7fd56ba, underlay_address: fd00:1122:3344:101::e, filesystem_pool: Some(ZpoolName { id: 2deff32e-1106-452e-a0e4-7f5633a4a1c4 (zpool), kind: External }), zone_type: Clickhouse { address: [fd00:1122:3344:101::e]:8123, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 2deff32e-1106-452e-a0e4-7f5633a4a1c4 (zpool), kind: External } } } }, OmicronZoneConfig { id: 2ddb8173-d078-4b99-bac5-ed06b4cf9def, underlay_address: fd00:1122:3344:101::f, filesystem_pool: Some(ZpoolName { id: fe61a674-6b99-47f9-b238-13f6d9a31093 (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::f]:17000 } }, OmicronZoneConfig { id: dbf1e583-6b72-47ae-8a84-3ce2331a120b, underlay_address: fd00:1122:3344:101::10, filesystem_pool: Some(ZpoolName { id: 753cff17-fbdf-48d1-960a-a1fe6d8c01ed (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::10]:17000 } }, OmicronZoneConfig { id: 8a6d7ab2-57bd-4341-9872-076f7f2d8bfd, underlay_address: fd00:1122:3344:101::11, filesystem_pool: Some(ZpoolName { id: 753cff17-fbdf-48d1-960a-a1fe6d8c01ed (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::11]:17000 } }, OmicronZoneConfig { id: 3d63d074-b018-4e87-8045-b03641d9d1f1, underlay_address: fd00:1122:3344:101::12, filesystem_pool: Some(ZpoolName { id: 2deff32e-1106-452e-a0e4-7f5633a4a1c4 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::12]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 2deff32e-1106-452e-a0e4-7f5633a4a1c4 (zpool), kind: External } } } }, OmicronZoneConfig { id: 02dc2ce9-7b01-425e-b378-1a4d032273cf, underlay_address: fd00:1122:3344:101::13, filesystem_pool: Some(ZpoolName { id: 903e5bdb-4fa3-466b-9cc6-cb6c1205c20b (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::13]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 903e5bdb-4fa3-466b-9cc6-cb6c1205c20b (zpool), kind: External } } } }, OmicronZoneConfig { id: 36357f09-d97f-4b2f-a4fd-16007d12067e, underlay_address: fd00:1122:3344:101::14, filesystem_pool: Some(ZpoolName { id: 8c0884cf-f16d-4d49-a677-c20655479168 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::14]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8c0884cf-f16d-4d49-a677-c20655479168 (zpool), kind: External } } } }, OmicronZoneConfig { id: 15a66369-82a9-40a2-a682-fbeb107fc5d7, underlay_address: fd00:1122:3344:101::15, filesystem_pool: Some(ZpoolName { id: 39cc409d-9820-46b7-b64f-11f4d431b612 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::15]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 39cc409d-9820-46b7-b64f-11f4d431b612 (zpool), kind: External } } } }, OmicronZoneConfig { id: c3f1e562-0d21-4f0e-863a-66d20b9899f7, underlay_address: fd00:1122:3344:101::16, filesystem_pool: Some(ZpoolName { id: 753cff17-fbdf-48d1-960a-a1fe6d8c01ed (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::16]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 753cff17-fbdf-48d1-960a-a1fe6d8c01ed (zpool), kind: External } } } }, OmicronZoneConfig { id: d14a7041-b90b-487e-acaa-0c8f6daa795f, underlay_address: fd00:1122:3344:101::17, filesystem_pool: Some(ZpoolName { id: fe61a674-6b99-47f9-b238-13f6d9a31093 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::17]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fe61a674-6b99-47f9-b238-13f6d9a31093 (zpool), kind: External } } } }, OmicronZoneConfig { id: 68525002-65a3-43fc-8e3f-0c38a0d081ff, underlay_address: fd00:1122:3344:101::18, filesystem_pool: Some(ZpoolName { id: 9b7eedae-40f6-4e96-bd9d-f89d715d8ad7 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::18]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9b7eedae-40f6-4e96-bd9d-f89d715d8ad7 (zpool), kind: External } } } }, OmicronZoneConfig { id: 0ed664b9-9a6f-41dc-a5f5-b38df3412517, underlay_address: fd00:1122:3344:101::19, filesystem_pool: Some(ZpoolName { id: 1a002b08-6021-4882-a443-e176f8f87821 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::19]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 1a002b08-6021-4882-a443-e176f8f87821 (zpool), kind: External } } } }, OmicronZoneConfig { id: bed84857-bc98-412a-90cd-2b38b7020c71, underlay_address: fd00:1122:3344:101::1a, filesystem_pool: Some(ZpoolName { id: 2936c097-160d-4a3f-a5e8-a1c6ffbf1edb (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::1a]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 2936c097-160d-4a3f-a5e8-a1c6ffbf1edb (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:438 sled_address = [fd00:1122:3344:101::1]:12345
9532024-08-28T00:55:11.510ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:39426
9542024-08-28T00:55:11.512ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 7ad61349-3006-4e30-bd4f-4104affaacb7, dataset_name: DatasetName { pool_name: ZpoolName { id: 2deff32e-1106-452e-a0e4-7f5633a4a1c4 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:406
9552024-08-28T00:55:11.512ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 7ad61349-3006-4e30-bd4f-4104affaacb7, dataset_name: DatasetName { pool_name: ZpoolName { id: 2deff32e-1106-452e-a0e4-7f5633a4a1c4 (zpool), kind: External }, kind: InternalDns }, responder: ... }
9562024-08-28T00:55:11.538ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 4dfcfd32-ab45-42c7-9c2f-4d836b875b6a, dataset_name: DatasetName { pool_name: ZpoolName { id: 903e5bdb-4fa3-466b-9cc6-cb6c1205c20b (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:406
9572024-08-28T00:55:11.538ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 4dfcfd32-ab45-42c7-9c2f-4d836b875b6a, dataset_name: DatasetName { pool_name: ZpoolName { id: 903e5bdb-4fa3-466b-9cc6-cb6c1205c20b (zpool), kind: External }, kind: InternalDns }, responder: ... }
9582024-08-28T00:55:11.558ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 09ed9686-d85a-429a-baae-bf2b96abe530, dataset_name: DatasetName { pool_name: ZpoolName { id: 8c0884cf-f16d-4d49-a677-c20655479168 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:406
9592024-08-28T00:55:11.558ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 09ed9686-d85a-429a-baae-bf2b96abe530, dataset_name: DatasetName { pool_name: ZpoolName { id: 8c0884cf-f16d-4d49-a677-c20655479168 (zpool), kind: External }, kind: InternalDns }, responder: ... }
9602024-08-28T00:55:11.579ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 62d7b608-ad89-43a5-a5eb-30c5f510a264, dataset_name: DatasetName { pool_name: ZpoolName { id: 2deff32e-1106-452e-a0e4-7f5633a4a1c4 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:406
9612024-08-28T00:55:11.579ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 62d7b608-ad89-43a5-a5eb-30c5f510a264, dataset_name: DatasetName { pool_name: ZpoolName { id: 2deff32e-1106-452e-a0e4-7f5633a4a1c4 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
9622024-08-28T00:55:11.599ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c21bdc0c-57c3-4a54-aa95-85bc9627d5b9, dataset_name: DatasetName { pool_name: ZpoolName { id: 903e5bdb-4fa3-466b-9cc6-cb6c1205c20b (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:406
9632024-08-28T00:55:11.599ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: c21bdc0c-57c3-4a54-aa95-85bc9627d5b9, dataset_name: DatasetName { pool_name: ZpoolName { id: 903e5bdb-4fa3-466b-9cc6-cb6c1205c20b (zpool), kind: External }, kind: CockroachDb }, responder: ... }
9642024-08-28T00:55:11.620ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2f3a388e-ad90-4757-82e1-21643e753681, dataset_name: DatasetName { pool_name: ZpoolName { id: 8c0884cf-f16d-4d49-a677-c20655479168 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:406
9652024-08-28T00:55:11.620ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 2f3a388e-ad90-4757-82e1-21643e753681, dataset_name: DatasetName { pool_name: ZpoolName { id: 8c0884cf-f16d-4d49-a677-c20655479168 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
9662024-08-28T00:55:11.640ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: fdb1d73c-f437-4e37-b800-c198de5adf74, dataset_name: DatasetName { pool_name: ZpoolName { id: 39cc409d-9820-46b7-b64f-11f4d431b612 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:406
9672024-08-28T00:55:11.640ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: fdb1d73c-f437-4e37-b800-c198de5adf74, dataset_name: DatasetName { pool_name: ZpoolName { id: 39cc409d-9820-46b7-b64f-11f4d431b612 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
9682024-08-28T00:55:11.659ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: bab3ad8b-d8a3-4d57-816f-a33fdf2870fb, dataset_name: DatasetName { pool_name: ZpoolName { id: 753cff17-fbdf-48d1-960a-a1fe6d8c01ed (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:406
9692024-08-28T00:55:11.659ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: bab3ad8b-d8a3-4d57-816f-a33fdf2870fb, dataset_name: DatasetName { pool_name: ZpoolName { id: 753cff17-fbdf-48d1-960a-a1fe6d8c01ed (zpool), kind: External }, kind: CockroachDb }, responder: ... }
9702024-08-28T00:55:11.679ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: fa9e7556-b41e-4114-bf3f-e918820a9bcf, dataset_name: DatasetName { pool_name: ZpoolName { id: 2deff32e-1106-452e-a0e4-7f5633a4a1c4 (zpool), kind: External }, kind: ExternalDns }, responder: ... }) file = sled-storage/src/manager.rs:406
9712024-08-28T00:55:11.679ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: fa9e7556-b41e-4114-bf3f-e918820a9bcf, dataset_name: DatasetName { pool_name: ZpoolName { id: 2deff32e-1106-452e-a0e4-7f5633a4a1c4 (zpool), kind: External }, kind: ExternalDns }, responder: ... }
9722024-08-28T00:55:11.760ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2cae9f85-8043-498f-af8e-a4bc9724ac55, dataset_name: DatasetName { pool_name: ZpoolName { id: 903e5bdb-4fa3-466b-9cc6-cb6c1205c20b (zpool), kind: External }, kind: ExternalDns }, responder: ... }) file = sled-storage/src/manager.rs:406
9732024-08-28T00:55:11.760ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 2cae9f85-8043-498f-af8e-a4bc9724ac55, dataset_name: DatasetName { pool_name: ZpoolName { id: 903e5bdb-4fa3-466b-9cc6-cb6c1205c20b (zpool), kind: External }, kind: ExternalDns }, responder: ... }
9742024-08-28T00:55:11.839ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: b5e20b06-936c-44f5-a47b-458ef7fd56ba, dataset_name: DatasetName { pool_name: ZpoolName { id: 2deff32e-1106-452e-a0e4-7f5633a4a1c4 (zpool), kind: External }, kind: Clickhouse }, responder: ... }) file = sled-storage/src/manager.rs:406
9752024-08-28T00:55:11.839ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: b5e20b06-936c-44f5-a47b-458ef7fd56ba, dataset_name: DatasetName { pool_name: ZpoolName { id: 2deff32e-1106-452e-a0e4-7f5633a4a1c4 (zpool), kind: External }, kind: Clickhouse }, responder: ... }
9762024-08-28T00:55:11.922ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 3d63d074-b018-4e87-8045-b03641d9d1f1, dataset_name: DatasetName { pool_name: ZpoolName { id: 2deff32e-1106-452e-a0e4-7f5633a4a1c4 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:406
9772024-08-28T00:55:11.922ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 3d63d074-b018-4e87-8045-b03641d9d1f1, dataset_name: DatasetName { pool_name: ZpoolName { id: 2deff32e-1106-452e-a0e4-7f5633a4a1c4 (zpool), kind: External }, kind: Crucible }, responder: ... }
9782024-08-28T00:55:12.004ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 02dc2ce9-7b01-425e-b378-1a4d032273cf, dataset_name: DatasetName { pool_name: ZpoolName { id: 903e5bdb-4fa3-466b-9cc6-cb6c1205c20b (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:406
9792024-08-28T00:55:12.004ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 02dc2ce9-7b01-425e-b378-1a4d032273cf, dataset_name: DatasetName { pool_name: ZpoolName { id: 903e5bdb-4fa3-466b-9cc6-cb6c1205c20b (zpool), kind: External }, kind: Crucible }, responder: ... }
9802024-08-28T00:55:12.082ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 36357f09-d97f-4b2f-a4fd-16007d12067e, dataset_name: DatasetName { pool_name: ZpoolName { id: 8c0884cf-f16d-4d49-a677-c20655479168 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:406
9812024-08-28T00:55:12.082ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 36357f09-d97f-4b2f-a4fd-16007d12067e, dataset_name: DatasetName { pool_name: ZpoolName { id: 8c0884cf-f16d-4d49-a677-c20655479168 (zpool), kind: External }, kind: Crucible }, responder: ... }
9822024-08-28T00:55:12.166ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 15a66369-82a9-40a2-a682-fbeb107fc5d7, dataset_name: DatasetName { pool_name: ZpoolName { id: 39cc409d-9820-46b7-b64f-11f4d431b612 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:406
9832024-08-28T00:55:12.166ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 15a66369-82a9-40a2-a682-fbeb107fc5d7, dataset_name: DatasetName { pool_name: ZpoolName { id: 39cc409d-9820-46b7-b64f-11f4d431b612 (zpool), kind: External }, kind: Crucible }, responder: ... }
9842024-08-28T00:55:12.243ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c3f1e562-0d21-4f0e-863a-66d20b9899f7, dataset_name: DatasetName { pool_name: ZpoolName { id: 753cff17-fbdf-48d1-960a-a1fe6d8c01ed (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:406
9852024-08-28T00:55:12.243ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: c3f1e562-0d21-4f0e-863a-66d20b9899f7, dataset_name: DatasetName { pool_name: ZpoolName { id: 753cff17-fbdf-48d1-960a-a1fe6d8c01ed (zpool), kind: External }, kind: Crucible }, responder: ... }
9862024-08-28T00:55:12.316ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: d14a7041-b90b-487e-acaa-0c8f6daa795f, dataset_name: DatasetName { pool_name: ZpoolName { id: fe61a674-6b99-47f9-b238-13f6d9a31093 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:406
9872024-08-28T00:55:12.316ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: d14a7041-b90b-487e-acaa-0c8f6daa795f, dataset_name: DatasetName { pool_name: ZpoolName { id: fe61a674-6b99-47f9-b238-13f6d9a31093 (zpool), kind: External }, kind: Crucible }, responder: ... }
9882024-08-28T00:55:12.382ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 68525002-65a3-43fc-8e3f-0c38a0d081ff, dataset_name: DatasetName { pool_name: ZpoolName { id: 9b7eedae-40f6-4e96-bd9d-f89d715d8ad7 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:406
9892024-08-28T00:55:12.383ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 68525002-65a3-43fc-8e3f-0c38a0d081ff, dataset_name: DatasetName { pool_name: ZpoolName { id: 9b7eedae-40f6-4e96-bd9d-f89d715d8ad7 (zpool), kind: External }, kind: Crucible }, responder: ... }
9902024-08-28T00:55:12.453ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 0ed664b9-9a6f-41dc-a5f5-b38df3412517, dataset_name: DatasetName { pool_name: ZpoolName { id: 1a002b08-6021-4882-a443-e176f8f87821 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:406
9912024-08-28T00:55:12.453ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 0ed664b9-9a6f-41dc-a5f5-b38df3412517, dataset_name: DatasetName { pool_name: ZpoolName { id: 1a002b08-6021-4882-a443-e176f8f87821 (zpool), kind: External }, kind: Crucible }, responder: ... }
9922024-08-28T00:55:12.519ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: bed84857-bc98-412a-90cd-2b38b7020c71, dataset_name: DatasetName { pool_name: ZpoolName { id: 2936c097-160d-4a3f-a5e8-a1c6ffbf1edb (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:406
9932024-08-28T00:55:12.520ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: bed84857-bc98-412a-90cd-2b38b7020c71, dataset_name: DatasetName { pool_name: ZpoolName { id: 2936c097-160d-4a3f-a5e8-a1c6ffbf1edb (zpool), kind: External }, kind: Crucible }, responder: ... }
9942024-08-28T00:55:12.585ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
9952024-08-28T00:55:12.585ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/1d8f47b2-ceb5-4768-b5e0-de73f9c6288a/config/omicron-zones.json file = common/src/ledger.rs:177
9962024-08-28T00:55:12.586ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/2701509e-5560-4f63-bb12-b820fcc5e4cb/config/omicron-zones.json file = common/src/ledger.rs:177
9972024-08-28T00:55:12.586ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
9982024-08-28T00:55:12.697ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
9992024-08-28T00:55:12.699ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
10002024-08-28T00:55:12.725ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
10012024-08-28T00:55:12.726ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
10022024-08-28T00:55:12.726ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
10032024-08-28T00:55:12.765ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
10042024-08-28T00:55:12.765ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
10052024-08-28T00:55:12.765ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
10062024-08-28T00:55:12.765ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
10072024-08-28T00:55:12.765ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
10082024-08-28T00:55:12.921ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
10092024-08-28T00:55:13.001ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
10102024-08-28T00:55:13.001ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
10112024-08-28T00:55:13.002ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
10122024-08-28T00:55:13.002ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
10132024-08-28T00:55:13.002ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
10142024-08-28T00:55:13.002ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
10152024-08-28T00:55:13.002ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
10162024-08-28T00:55:13.002ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
10172024-08-28T00:55:13.003ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba file = illumos-utils/src/zone.rs:316
10182024-08-28T00:55:13.009ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def file = illumos-utils/src/zone.rs:316
10192024-08-28T00:55:13.014ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b file = illumos-utils/src/zone.rs:316
10202024-08-28T00:55:13.019ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1 file = illumos-utils/src/zone.rs:316
10212024-08-28T00:55:13.025ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf file = illumos-utils/src/zone.rs:316
10222024-08-28T00:55:13.031ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26 file = illumos-utils/src/zone.rs:316
10232024-08-28T00:55:13.036ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7 file = illumos-utils/src/zone.rs:316
10242024-08-28T00:55:13.187ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba file = illumos-utils/src/zone.rs:354
10252024-08-28T00:55:13.193ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def file = illumos-utils/src/zone.rs:354
10262024-08-28T00:55:13.198ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b file = illumos-utils/src/zone.rs:354
10272024-08-28T00:55:13.203ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1 file = illumos-utils/src/zone.rs:354
10282024-08-28T00:55:13.208ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf file = illumos-utils/src/zone.rs:354
10292024-08-28T00:55:13.212ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26 file = illumos-utils/src/zone.rs:354
10302024-08-28T00:55:13.217ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7 file = illumos-utils/src/zone.rs:354
10312024-08-28T00:55:13.222ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f file = illumos-utils/src/zone.rs:316
10322024-08-28T00:55:13.228ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517 file = illumos-utils/src/zone.rs:316
10332024-08-28T00:55:13.234ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd file = illumos-utils/src/zone.rs:316
10342024-08-28T00:55:13.240ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7 file = illumos-utils/src/zone.rs:316
10352024-08-28T00:55:13.245ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff file = illumos-utils/src/zone.rs:316
10362024-08-28T00:55:13.251ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71 file = illumos-utils/src/zone.rs:316
10372024-08-28T00:55:13.257ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e file = illumos-utils/src/zone.rs:316
10382024-08-28T00:55:13.316ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
10392024-08-28T00:55:13.316ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f file = illumos-utils/src/zone.rs:354
10402024-08-28T00:55:13.325ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517 file = illumos-utils/src/zone.rs:354
10412024-08-28T00:55:13.332ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd file = illumos-utils/src/zone.rs:354
10422024-08-28T00:55:13.340ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7 file = illumos-utils/src/zone.rs:354
10432024-08-28T00:55:13.347ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff file = illumos-utils/src/zone.rs:354
10442024-08-28T00:55:13.354ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71 file = illumos-utils/src/zone.rs:354
10452024-08-28T00:55:13.361ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e file = illumos-utils/src/zone.rs:354
10462024-08-28T00:55:13.445ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
10472024-08-28T00:55:13.519ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
10482024-08-28T00:55:13.624ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
10492024-08-28T00:55:13.627ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
10502024-08-28T00:55:13.627ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
10512024-08-28T00:55:13.684ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte1", ip: 172.30.2.7, mac: MacAddr6([168, 64, 37, 255, 191, 68]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
10522024-08-28T00:55:13.685ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1235 zone_type = nexus
10532024-08-28T00:55:13.747ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
10542024-08-28T00:55:13.748ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
10552024-08-28T00:55:13.748ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
10562024-08-28T00:55:13.803ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte2", ip: 172.30.1.5, mac: MacAddr6([168, 64, 37, 255, 202, 132]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.1.1 } }) }
10572024-08-28T00:55:13.803ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1235 zone_type = external_dns
10582024-08-28T00:55:13.805ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
10592024-08-28T00:55:13.805ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
10602024-08-28T00:55:13.864ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte3", ip: 172.30.2.5, mac: MacAddr6([168, 64, 37, 255, 220, 190]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
10612024-08-28T00:55:13.864ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1235 zone_type = nexus
10622024-08-28T00:55:13.865ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
10632024-08-28T00:55:13.865ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
10642024-08-28T00:55:13.923ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte4", ip: 172.30.1.6, mac: MacAddr6([168, 64, 37, 255, 212, 38]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.1.1 } }) }
10652024-08-28T00:55:13.923ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1235 zone_type = external_dns
10662024-08-28T00:55:13.923ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.1.168 file = clients/dpd-client/src/lib.rs:83
10672024-08-28T00:55:13.923ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.1.168 file = clients/dpd-client/src/lib.rs:90
10682024-08-28T00:55:13.925ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
10692024-08-28T00:55:13.925ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
10702024-08-28T00:55:13.981ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte5", ip: 172.30.2.6, mac: MacAddr6([168, 64, 37, 255, 209, 34]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
10712024-08-28T00:55:13.981ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1235 zone_type = nexus
10722024-08-28T00:55:13.981ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.1.168 file = clients/dpd-client/src/lib.rs:118
10732024-08-28T00:55:13.983ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.1.164 file = clients/dpd-client/src/lib.rs:83
10742024-08-28T00:55:13.983ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.1.164 file = clients/dpd-client/src/lib.rs:90
10752024-08-28T00:55:13.983ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
10762024-08-28T00:55:14.028ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.1.166 file = clients/dpd-client/src/lib.rs:83
10772024-08-28T00:55:14.028ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.1.166 file = clients/dpd-client/src/lib.rs:90
10782024-08-28T00:55:14.029ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.1.164 file = clients/dpd-client/src/lib.rs:118
10792024-08-28T00:55:14.032ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
10802024-08-28T00:55:14.032ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.1.165 file = clients/dpd-client/src/lib.rs:83
10812024-08-28T00:55:14.032ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.1.165 file = clients/dpd-client/src/lib.rs:90
10822024-08-28T00:55:14.034ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.1.166 file = clients/dpd-client/src/lib.rs:118
10832024-08-28T00:55:14.073ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.1.165 file = clients/dpd-client/src/lib.rs:118
10842024-08-28T00:55:14.076ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.1.167 file = clients/dpd-client/src/lib.rs:83
10852024-08-28T00:55:14.076ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.1.167 file = clients/dpd-client/src/lib.rs:90
10862024-08-28T00:55:14.076ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
10872024-08-28T00:55:14.077ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
10882024-08-28T00:55:14.077ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166 file = illumos-utils/src/zone.rs:316
10892024-08-28T00:55:14.118ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf file = illumos-utils/src/zone.rs:316
10902024-08-28T00:55:14.166ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.1.167 file = clients/dpd-client/src/lib.rs:118
10912024-08-28T00:55:14.170ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
10922024-08-28T00:55:14.170ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166 file = illumos-utils/src/zone.rs:354
10932024-08-28T00:55:14.180ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739 file = illumos-utils/src/zone.rs:316
10942024-08-28T00:55:14.190ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf file = illumos-utils/src/zone.rs:354
10952024-08-28T00:55:14.208ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55 file = illumos-utils/src/zone.rs:316
10962024-08-28T00:55:14.259ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739 file = illumos-utils/src/zone.rs:354
10972024-08-28T00:55:14.270ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55 file = illumos-utils/src/zone.rs:354
10982024-08-28T00:55:14.283ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d file = illumos-utils/src/zone.rs:316
10992024-08-28T00:55:14.323ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d file = illumos-utils/src/zone.rs:354
11002024-08-28T00:55:16.668ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService11'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::f'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::f'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11012024-08-28T00:55:16.668ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def
11022024-08-28T00:55:16.833ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService18'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::19'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_1a002b08-6021-4882-a443-e176f8f87821/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::19'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='51b445ab-b8b2-4a68-af5c-04778e86fb9b'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11032024-08-28T00:55:16.833ZINFOSledAgent (ServiceManager): Booting oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517
11042024-08-28T00:55:16.848ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService23'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::14'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_8c0884cf-f16d-4d49-a677-c20655479168/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::14'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='f1255532-8fac-4eb3-9313-4da6562c9906'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11052024-08-28T00:55:16.848ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService13'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::12'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_2deff32e-1106-452e-a0e4-7f5633a4a1c4/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::12'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='11b1eb68-6159-4fa0-a42a-e318b6ed5c6c'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11062024-08-28T00:55:16.849ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService14'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::13'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_903e5bdb-4fa3-466b-9cc6-cb6c1205c20b/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::13'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='d1398d7b-3800-450f-922a-e9a73f7ecf60'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11072024-08-28T00:55:16.849ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService16'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::15'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_39cc409d-9820-46b7-b64f-11f4d431b612/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::15'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='cd9a8b57-4f60-4ba4-b850-449ceb95db20'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11082024-08-28T00:55:16.849ZINFOSledAgent (ServiceManager): Booting oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e
11092024-08-28T00:55:16.860ZINFOSledAgent (ServiceManager): Booting oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1
11102024-08-28T00:55:16.871ZINFOSledAgent (ServiceManager): Booting oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf
11112024-08-28T00:55:16.896ZINFOSledAgent (ServiceManager): Booting oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7
11122024-08-28T00:55:16.932ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService21'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::18'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_9b7eedae-40f6-4e96-bd9d-f89d715d8ad7/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::18'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='7ab958e8-173f-4658-8c9a-581833916c5f'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11132024-08-28T00:55:16.933ZINFOSledAgent (ServiceManager): Booting oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff
11142024-08-28T00:55:16.960ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService17'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::17'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_fe61a674-6b99-47f9-b238-13f6d9a31093/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::17'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='b9fda28f-cfa9-4a5d-bb79-636ac1de0765'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11152024-08-28T00:55:16.961ZINFOSledAgent (ServiceManager): Booting oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f
11162024-08-28T00:55:17.015ZINFOSledAgent (ServiceManager): Profile for oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService15'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::d'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/oximeter"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="id" value='4fe6d00e-187b-4dd4-9562-e20645ddec26'/> <propval type="astring" name="address" value='[fd00:1122:3344:101::d]:12223'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11172024-08-28T00:55:17.015ZINFOSledAgent (ServiceManager): Booting oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26
11182024-08-28T00:55:17.029ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService22'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::1a'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_2936c097-160d-4a3f-a5e8-a1c6ffbf1edb/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::1a'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='a0f2577d-5017-4d02-b0f7-76e293369c1a'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11192024-08-28T00:55:17.030ZINFOSledAgent (ServiceManager): Booting oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71
11202024-08-28T00:55:17.146ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def") failed: Property not found. retry in 55.545703ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def
11212024-08-28T00:55:17.157ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e") failed: Property not found. retry in 47.454238ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e
11222024-08-28T00:55:17.167ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf") failed: Property not found. retry in 56.598054ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf
11232024-08-28T00:55:17.176ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517") failed: Property not found. retry in 55.789062ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517
11242024-08-28T00:55:17.186ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1") failed: Property not found. retry in 47.535717ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1
11252024-08-28T00:55:17.214ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def") failed: Property not found. retry in 50.315394ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def
11262024-08-28T00:55:17.245ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7") failed: Property not found. retry in 28.706579ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7
11272024-08-28T00:55:17.374ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff") failed: Property not found. retry in 71.056956ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff
11282024-08-28T00:55:17.399ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e") failed: Property not found. retry in 102.121993ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e
11292024-08-28T00:55:17.653ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def") failed: Property not found. retry in 246.452801ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def
11302024-08-28T00:55:17.809ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1") failed: Property not found. retry in 94.107236ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1
11312024-08-28T00:55:18.058ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf") failed: Property not found. retry in 84.817206ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf
11322024-08-28T00:55:18.271ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26") failed: Property not found. retry in 49.5764ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26
11332024-08-28T00:55:18.349ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7") failed: Property not found. retry in 80.683743ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7
11342024-08-28T00:55:18.402ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f") failed: Property not found. retry in 43.632537ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f
11352024-08-28T00:55:18.510ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517") failed: Property not found. retry in 113.145105ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517
11362024-08-28T00:55:18.791ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff") failed: Property not found. retry in 127.576695ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff
11372024-08-28T00:55:18.821ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71") failed: Property not found. retry in 28.537648ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71
11382024-08-28T00:55:18.883ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e") failed: Property not found. retry in 166.182435ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e
11392024-08-28T00:55:18.936ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def") failed: Property not found. retry in 340.35814ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def
11402024-08-28T00:55:18.976ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1") failed: Property not found. retry in 163.560756ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1
11412024-08-28T00:55:18.992ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf") failed: Property not found. retry in 194.867583ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf
11422024-08-28T00:55:19.010ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26") failed: Property not found. retry in 77.101136ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26
11432024-08-28T00:55:19.028ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7") failed: Property not found. retry in 171.287995ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7
11442024-08-28T00:55:19.044ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f") failed: Property not found. retry in 98.296507ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f
11452024-08-28T00:55:19.059ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517") failed: Property not found. retry in 237.734475ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517
11462024-08-28T00:55:19.079ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff") failed: Property not found. retry in 125.06814ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff
11472024-08-28T00:55:19.095ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71") failed: Property not found. retry in 147.042869ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71
11482024-08-28T00:55:19.109ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e") failed: Property not found. retry in 504.73019ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e
11492024-08-28T00:55:19.115ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService12'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::10'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::10'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11502024-08-28T00:55:19.133ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26") failed: Property not found. retry in 188.680343ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26
11512024-08-28T00:55:19.146ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService19'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::11'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::11'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11522024-08-28T00:55:19.148ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b
11532024-08-28T00:55:19.205ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1") failed: Property not found. retry in 288.331304ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1
11542024-08-28T00:55:19.220ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf") failed: Property not found. retry in 337.835008ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf
11552024-08-28T00:55:19.234ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7") failed: Property not found. retry in 236.168259ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7
11562024-08-28T00:55:19.248ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f") failed: Property not found. retry in 113.477044ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f
11572024-08-28T00:55:19.248ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd
11582024-08-28T00:55:19.322ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff") failed: Property not found. retry in 261.974191ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff
11592024-08-28T00:55:19.336ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71") failed: Property not found. retry in 184.531945ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71
11602024-08-28T00:55:19.352ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def") failed: Property not found. retry in 722.038863ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def
11612024-08-28T00:55:19.367ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26") failed: Property not found. retry in 401.616071ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26
11622024-08-28T00:55:19.383ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f") failed: Property not found. retry in 250.818633ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f
11632024-08-28T00:55:19.399ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517") failed: Property not found. retry in 306.734184ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517
11642024-08-28T00:55:19.400ZINFOSledAgent (ServiceManager): Profile for oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService25'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::8'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte2'/> <propval type="astring" name="gateway" value='172.30.1.1'/> <propval type="astring" name="ip" value='172.30.1.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/external_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::8]:5353'/> <propval type="astring" name="dns_address" value='172.30.1.5:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11652024-08-28T00:55:19.404ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService20'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::16'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_753cff17-fbdf-48d1-960a-a1fe6d8c01ed/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::16'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='9c278140-a7f1-4a01-859e-11c18d607e86'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11662024-08-28T00:55:19.408ZINFOSledAgent (ServiceManager): Booting oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7
11672024-08-28T00:55:24.259ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff") failed: Property not found. retry in 617.416757ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff
11682024-08-28T00:55:24.284ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:35592
11692024-08-28T00:55:24.295ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71") failed: Property not found. retry in 233.2469ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71
11702024-08-28T00:55:24.335ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e") failed: Property not found. retry in 964.213845ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e
11712024-08-28T00:55:24.336ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8644 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:35592 req_id = fc645e03-899b-4e12-b6bd-b566d33fe1a8 response_code = 200 uri = /baseboard
11722024-08-28T00:55:24.365ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def") failed: Property not found. retry in 1.14333947s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def
11732024-08-28T00:55:24.365ZINFOSledAgent (ServiceManager): Booting oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf
11742024-08-28T00:55:24.382ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b") failed: Property not found. retry in 53.787409ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b
11752024-08-28T00:55:24.396ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1") failed: Property not found. retry in 602.040998ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1
11762024-08-28T00:55:24.408ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf") failed: Property not found. retry in 582.947338ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf
11772024-08-28T00:55:24.423ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26") failed: Property not found. retry in 636.618389ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26
11782024-08-28T00:55:24.434ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7") failed: Property not found. retry in 1.081196674s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7
11792024-08-28T00:55:24.435ZINFOSledAgent (ServiceManager): Profile for oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService27'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::9'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte4'/> <propval type="astring" name="gateway" value='172.30.1.1'/> <propval type="astring" name="ip" value='172.30.1.6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/external_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::9]:5353'/> <propval type="astring" name="dns_address" value='172.30.1.6:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11802024-08-28T00:55:24.448ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f") failed: Property not found. retry in 919.385613ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f
11812024-08-28T00:55:24.461ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517") failed: Property not found. retry in 1.171494956s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517
11822024-08-28T00:55:24.474ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd") failed: Property not found. retry in 66.329938ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd
11832024-08-28T00:55:24.488ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b") failed: Property not found. retry in 111.211585ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b
11842024-08-28T00:55:24.488ZINFOSledAgent (ServiceManager): Booting oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55
11852024-08-28T00:55:24.496ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService28'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::b'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte5'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11862024-08-28T00:55:24.496ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService24'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::c'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte1'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.7'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11872024-08-28T00:55:24.496ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService26'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::a'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte3'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11882024-08-28T00:55:24.496ZINFOSledAgent (ServiceManager): Booting oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166
11892024-08-28T00:55:24.504ZINFOSledAgent (ServiceManager): Booting oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739
11902024-08-28T00:55:24.510ZINFOSledAgent (ServiceManager): Booting oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d
11912024-08-28T00:55:24.541ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71") failed: Property not found. retry in 533.790059ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71
11922024-08-28T00:55:24.555ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd") failed: Property not found. retry in 125.61976ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd
11932024-08-28T00:55:24.568ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7") failed: Property not found. retry in 43.318256ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7
11942024-08-28T00:55:24.612ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b") failed: Property not found. retry in 137.402603ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b
11952024-08-28T00:55:24.624ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7") failed: Property not found. retry in 111.294495ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7
11962024-08-28T00:55:24.678ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf") failed: Property not found. retry in 42.927978ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf
11972024-08-28T00:55:24.695ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd") failed: Property not found. retry in 168.476932ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd
11982024-08-28T00:55:24.721ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166") failed: Property not found. retry in 41.048388ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166
11992024-08-28T00:55:24.733ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739") failed: Property not found. retry in 45.948368ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739
12002024-08-28T00:55:24.746ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf") failed: Property not found. retry in 99.559778ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf
12012024-08-28T00:55:24.758ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55") failed: Property not found. retry in 62.794256ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55
12022024-08-28T00:55:24.768ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7") failed: Property not found. retry in 122.941744ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7
12032024-08-28T00:55:24.779ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166") failed: Property not found. retry in 141.912046ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166
12042024-08-28T00:55:24.790ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b") failed: Property not found. retry in 546.819987ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b
12052024-08-28T00:55:24.800ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d") failed: Property not found. retry in 71.257359ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d
12062024-08-28T00:55:24.809ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739") failed: Property not found. retry in 121.451068ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739
12072024-08-28T00:55:24.833ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55") failed: Property not found. retry in 73.091232ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55
12082024-08-28T00:55:24.858ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf") failed: Property not found. retry in 240.116238ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf
12092024-08-28T00:55:24.875ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd") failed: Property not found. retry in 466.682725ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd
12102024-08-28T00:55:24.886ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d") failed: Property not found. retry in 149.67653ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d
12112024-08-28T00:55:24.896ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff") failed: Property not found. retry in 1.287866322s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff
12122024-08-28T00:55:24.906ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7") failed: Property not found. retry in 310.304486ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7
12132024-08-28T00:55:24.916ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55") failed: Property not found. retry in 115.585596ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55
12142024-08-28T00:55:24.932ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166") failed: Property not found. retry in 297.142771ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166
12152024-08-28T00:55:24.944ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739") failed: Property not found. retry in 166.195884ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739
12162024-08-28T00:55:25.003ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf") failed: Property not found. retry in 1.445618931s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf
12172024-08-28T00:55:25.019ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1") failed: Property not found. retry in 852.74895ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1
12182024-08-28T00:55:25.050ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55") failed: Property not found. retry in 536.554281ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55
12192024-08-28T00:55:25.065ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d") failed: Property not found. retry in 197.21968ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d
12202024-08-28T00:55:25.076ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26") failed: Property not found. retry in 1.326943876s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26
12212024-08-28T00:55:25.088ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71") failed: Property not found. retry in 556.772439ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71
12222024-08-28T00:55:25.111ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf") failed: Property not found. retry in 347.220974ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf
12232024-08-28T00:55:25.124ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739") failed: Property not found. retry in 352.57856ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739
12242024-08-28T00:55:25.229ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7") failed: Property not found. retry in 872.371682ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7
12252024-08-28T00:55:25.247ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166") failed: Property not found. retry in 328.172535ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166
12262024-08-28T00:55:25.274ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d") failed: Property not found. retry in 237.951844ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d
12272024-08-28T00:55:25.310ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e") failed: Property not found. retry in 877.224427ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e
12282024-08-28T00:55:25.348ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b") failed: Property not found. retry in 936.556117ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b
12292024-08-28T00:55:25.359ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd") failed: Property not found. retry in 1.199614109s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd
12302024-08-28T00:55:25.378ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f") failed: Property not found. retry in 626.571987ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f
12312024-08-28T00:55:25.471ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf") failed: Property not found. retry in 461.961347ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf
12322024-08-28T00:55:25.490ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739") failed: Property not found. retry in 648.653256ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739
12332024-08-28T00:55:25.522ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def") failed: Property not found. retry in 1.151623251s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def
12342024-08-28T00:55:25.532ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7") failed: Property not found. retry in 866.018231ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7
12352024-08-28T00:55:25.543ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d") failed: Property not found. retry in 917.852047ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d
12362024-08-28T00:55:25.587ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166") failed: Property not found. retry in 727.85469ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166
12372024-08-28T00:55:25.596ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55") failed: Property not found. retry in 1.005790953s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55
12382024-08-28T00:55:25.644ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517") failed: Property not found. retry in 1.114371636s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517
12392024-08-28T00:55:25.657ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71") failed: Property not found. retry in 1.306394095s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71
12402024-08-28T00:55:25.884ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1") failed: Property not found. retry in 1.168313239s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1
12412024-08-28T00:55:25.947ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf") failed: Property not found. retry in 1.355874852s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf
12422024-08-28T00:55:26.019ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f") failed: Property not found. retry in 1.118838881s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f
12432024-08-28T00:55:26.114ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7") failed: Property not found. retry in 1.485237656s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7
12442024-08-28T00:55:26.153ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739") failed: Property not found. retry in 973.827906ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739
12452024-08-28T00:55:26.197ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff") failed: Property not found. retry in 1.138906531s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff
12462024-08-28T00:55:26.208ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e") failed: Property not found. retry in 562.346746ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e
12472024-08-28T00:55:26.298ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b") failed: Property not found. retry in 860.852032ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b
12482024-08-28T00:55:26.327ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166") failed: Property not found. retry in 866.781079ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166
12492024-08-28T00:55:26.412ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7") failed: Property not found. retry in 868.749211ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7
12502024-08-28T00:55:26.424ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26") failed: Property not found. retry in 1.441203789s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26
12512024-08-28T00:55:26.462ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf") failed: Property not found. retry in 666.822079ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf
12522024-08-28T00:55:26.474ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d") failed: Property not found. retry in 862.602534ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d
12532024-08-28T00:55:26.574ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd") failed: Property not found. retry in 1.45985676s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd
12542024-08-28T00:55:26.615ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55") failed: Property not found. retry in 1.066505799s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55
12552024-08-28T00:55:26.682ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def") failed: Property not found. retry in 1.03352808s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def
12562024-08-28T00:55:26.768ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517") failed: Property not found. retry in 663.104336ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517
12572024-08-28T00:55:26.778ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e") failed: Property not found. retry in 694.946745ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e
12582024-08-28T00:55:26.974ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71") failed: Property not found. retry in 1.273727349s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71
12592024-08-28T00:55:27.063ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1") failed: Property not found. retry in 648.06675ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1
12602024-08-28T00:55:27.137ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739") failed: Property not found. retry in 988.322849ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739
12612024-08-28T00:55:27.144ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf") failed: Property not found. retry in 1.059019995s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf
12622024-08-28T00:55:27.152ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f") failed: Property not found. retry in 949.53443ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f
12632024-08-28T00:55:27.168ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b") failed: Property not found. retry in 994.782035ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b
12642024-08-28T00:55:27.202ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166") failed: Property not found. retry in 1.078556846s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166
12652024-08-28T00:55:27.289ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7") failed: Property not found. retry in 1.088476368s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7
12662024-08-28T00:55:27.313ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf") failed: Property not found. retry in 1.331797128s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf
12672024-08-28T00:55:27.345ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff") failed: Property not found. retry in 1.076286232s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff
12682024-08-28T00:55:27.353ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d") failed: Property not found. retry in 705.080456ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d
12692024-08-28T00:55:27.440ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517") failed: Property not found. retry in 1.113318902s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517
12702024-08-28T00:55:27.482ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e") failed: Property not found. retry in 1.479018406s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e
12712024-08-28T00:55:27.609ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7") failed: Property not found. retry in 679.550843ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7
12722024-08-28T00:55:27.690ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55") failed: Property not found. retry in 837.006868ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55
12732024-08-28T00:55:27.717ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1") failed: Property not found. retry in 1.196478646s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1
12742024-08-28T00:55:27.724ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def") failed: Property not found. retry in 823.81653ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def
12752024-08-28T00:55:27.875ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26") failed: Property not found. retry in 1.09357151s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26
12762024-08-28T00:55:28.061ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd") failed: Property not found. retry in 561.355832ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd
12772024-08-28T00:55:28.071ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d") failed: Property not found. retry in 1.366184568s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d
12782024-08-28T00:55:28.113ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f") failed: Property not found. retry in 1.039703995s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f
12792024-08-28T00:55:28.137ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739") failed: Property not found. retry in 605.76147ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739
12802024-08-28T00:55:28.176ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b") failed: Property not found. retry in 700.949386ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b
12812024-08-28T00:55:28.217ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf") failed: Property not found. retry in 896.076308ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf
12822024-08-28T00:55:28.259ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71") failed: Property not found. retry in 575.800881ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71
12832024-08-28T00:55:28.289ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166") failed: Property not found. retry in 1.493727432s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166
12842024-08-28T00:55:28.297ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7") failed: Property not found. retry in 1.183056396s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7
12852024-08-28T00:55:28.386ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7") failed: Property not found. retry in 517.052026ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7
12862024-08-28T00:55:28.432ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff") failed: Property not found. retry in 626.21989ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff
12872024-08-28T00:55:28.536ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55") failed: Property not found. retry in 1.38931307s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55
12882024-08-28T00:55:28.557ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def") failed: Property not found. retry in 956.591003ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def
12892024-08-28T00:55:28.564ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517") failed: Property not found. retry in 788.56ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517
12902024-08-28T00:55:28.631ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd") failed: Property not found. retry in 849.44908ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd
12912024-08-28T00:55:28.654ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf") failed: Property not found. retry in 782.62072ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf
12922024-08-28T00:55:28.752ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739") failed: Property not found. retry in 1.217221547s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739
12932024-08-28T00:55:28.845ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71") failed: Property not found. retry in 957.568692ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71
12942024-08-28T00:55:28.886ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b") failed: Property not found. retry in 824.094909ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b
12952024-08-28T00:55:28.912ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7") failed: Property not found. retry in 898.843767ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7
12962024-08-28T00:55:28.924ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1") failed: Property not found. retry in 634.671534ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1
12972024-08-28T00:55:29.094ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26") failed: Property not found. retry in 646.325889ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26
12982024-08-28T00:55:29.102ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff") failed: Property not found. retry in 1.462470518s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff
12992024-08-28T00:55:29.110ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e") failed: Property not found. retry in 617.630662ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e
13002024-08-28T00:55:29.123ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf") failed: Property not found. retry in 1.121541159s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf
13012024-08-28T00:55:29.162ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f") failed: Property not found. retry in 1.087216745s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f
13022024-08-28T00:55:29.363ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517") failed: Property not found. retry in 893.308468ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517
13032024-08-28T00:55:29.446ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf") failed: Property not found. retry in 1.4809958s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf
13042024-08-28T00:55:29.452ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d") failed: Property not found. retry in 1.058616187s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d
13052024-08-28T00:55:29.488ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7") failed: Property not found. retry in 902.179226ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7
13062024-08-28T00:55:29.495ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd") failed: Property not found. retry in 1.465644364s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd
13072024-08-28T00:55:29.521ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def") failed: Property not found. retry in 767.201858ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def
13082024-08-28T00:55:29.567ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1") failed: Property not found. retry in 1.133089666s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1
13092024-08-28T00:55:29.721ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b") failed: Property not found. retry in 739.629528ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b
13102024-08-28T00:55:29.736ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e") failed: Property not found. retry in 1.409713969s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e
13112024-08-28T00:55:29.748ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26") failed: Property not found. retry in 1.43603007s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26
13122024-08-28T00:55:29.793ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166") failed: Property not found. retry in 729.024246ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166
13132024-08-28T00:55:29.810ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71") failed: Property not found. retry in 1.404582655s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71
13142024-08-28T00:55:29.819ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7") failed: Property not found. retry in 1.378892562s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7
13152024-08-28T00:55:29.936ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55") failed: Property not found. retry in 1.2682213s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55
13162024-08-28T00:55:29.978ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739") failed: Property not found. retry in 632.445444ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739
13172024-08-28T00:55:30.149ZINFOSledAgent (ServiceManager): Profile for oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService10'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::e'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/clickhouse"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::e'/> <propval type="astring" name="listen_port" value='8123'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:1::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/clickhouse-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="clickhouse_address" value='[fd00:1122:3344:101::e]:8123'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::e]:8888'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13182024-08-28T00:55:30.154ZINFOSledAgent (ServiceManager): Booting oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba
13192024-08-28T00:55:30.255ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf") failed: Property not found. retry in 1.450024138s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf
13202024-08-28T00:55:30.263ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f") failed: Property not found. retry in 843.159839ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f
13212024-08-28T00:55:30.273ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517") failed: Property not found. retry in 1.150380248s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517
13222024-08-28T00:55:30.297ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def") failed: Property not found. retry in 540.485014ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def
13232024-08-28T00:55:30.389ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba") failed: Property not found. retry in 55.449051ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba
13242024-08-28T00:55:30.400ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7") failed: Property not found. retry in 1.034566953s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7
13252024-08-28T00:55:30.453ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba") failed: Property not found. retry in 103.942043ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba
13262024-08-28T00:55:30.469ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b") failed: Property not found. retry in 644.620742ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b
13272024-08-28T00:55:30.521ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d") failed: Property not found. retry in 1.145520176s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d
13282024-08-28T00:55:30.530ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166") failed: Property not found. retry in 1.458583329s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166
13292024-08-28T00:55:30.570ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba") failed: Property not found. retry in 104.787787ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba
13302024-08-28T00:55:30.580ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff") failed: Property not found. retry in 1.359878858s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff
13312024-08-28T00:55:30.621ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739") failed: Property not found. retry in 1.332537953s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739
13322024-08-28T00:55:30.686ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba") failed: Property not found. retry in 579.061729ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba
13332024-08-28T00:55:30.709ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1") failed: Property not found. retry in 915.281873ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1
13342024-08-28T00:55:30.847ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def") failed: Property not found. retry in 1.449741269s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def
13352024-08-28T00:55:30.936ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf") failed: Property not found. retry in 518.079896ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf
13362024-08-28T00:55:30.970ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd") failed: Property not found. retry in 1.105373695s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd
13372024-08-28T00:55:31.116ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f") failed: Property not found. retry in 1.117910246s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f
13382024-08-28T00:55:31.124ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b") failed: Property not found. retry in 1.198769761s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b
13392024-08-28T00:55:31.154ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e") failed: Property not found. retry in 792.782332ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e
13402024-08-28T00:55:31.194ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26") failed: Property not found. retry in 1.406355411s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26
13412024-08-28T00:55:31.206ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7") failed: Property not found. retry in 1.264138531s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7
13422024-08-28T00:55:31.214ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55") failed: Property not found. retry in 977.580402ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55
13432024-08-28T00:55:31.225ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71") failed: Property not found. retry in 1.174058748s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71
13442024-08-28T00:55:31.275ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba") failed: Property not found. retry in 457.736209ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba
13452024-08-28T00:55:31.433ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517") failed: Property not found. retry in 972.454373ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517
13462024-08-28T00:55:31.444ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7") failed: Property not found. retry in 705.251956ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7
13472024-08-28T00:55:31.464ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf") failed: Property not found. retry in 1.441628427s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf
13482024-08-28T00:55:31.635ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1") failed: Property not found. retry in 1.225917908s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1
13492024-08-28T00:55:31.676ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d") failed: Property not found. retry in 1.428874452s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d
13502024-08-28T00:55:31.714ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf") failed: Property not found. retry in 681.277393ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf
13512024-08-28T00:55:31.741ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba") failed: Property not found. retry in 603.460228ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba
13522024-08-28T00:55:31.950ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff") failed: Property not found. retry in 599.21595ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff
13532024-08-28T00:55:31.958ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e") failed: Property not found. retry in 537.998381ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e
13542024-08-28T00:55:31.967ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739") failed: Property not found. retry in 1.159108977s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739
13552024-08-28T00:55:31.998ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166") failed: Property not found. retry in 1.278123348s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166
13562024-08-28T00:55:32.085ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd") failed: Property not found. retry in 585.027569ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd
13572024-08-28T00:55:32.160ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7") failed: Property not found. retry in 1.444809846s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7
13582024-08-28T00:55:32.202ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55") failed: Property not found. retry in 945.174938ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55
13592024-08-28T00:55:32.244ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f") failed: Property not found. retry in 1.257573656s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f
13602024-08-28T00:55:32.308ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def") failed: Property not found. retry in 1.100914257s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def
13612024-08-28T00:55:32.333ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b") failed: Property not found. retry in 813.497121ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b
13622024-08-28T00:55:32.354ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba") failed: Property not found. retry in 1.312696944s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba
13632024-08-28T00:55:32.404ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf") failed: Property not found. retry in 997.140013ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf
13642024-08-28T00:55:32.411ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71") failed: Property not found. retry in 1.476739662s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71
13652024-08-28T00:55:32.419ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517") failed: Property not found. retry in 711.526531ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517
13662024-08-28T00:55:32.480ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7") failed: Property not found. retry in 673.960188ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7
13672024-08-28T00:55:32.505ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e") failed: Property not found. retry in 1.193807285s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e
13682024-08-28T00:55:32.558ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff") failed: Property not found. retry in 1.065662313s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff
13692024-08-28T00:55:32.609ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26") failed: Property not found. retry in 1.317779819s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26
13702024-08-28T00:55:32.682ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd") failed: Property not found. retry in 1.068624103s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd
13712024-08-28T00:55:32.872ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1") failed: Property not found. retry in 1.149213948s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1
13722024-08-28T00:55:32.916ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf") failed: Property not found. retry in 769.474258ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf
13732024-08-28T00:55:33.115ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d") failed: Property not found. retry in 1.46775779s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d
13742024-08-28T00:55:33.139ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739") failed: Property not found. retry in 572.507095ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739
13752024-08-28T00:55:33.148ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517") failed: Property not found. retry in 1.120907694s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517
13762024-08-28T00:55:33.155ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b") failed: Property not found. retry in 1.037429939s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b
13772024-08-28T00:55:33.163ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55") failed: Property not found. retry in 1.297022105s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55
13782024-08-28T00:55:33.172ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7") failed: Property not found. retry in 955.728829ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7
13792024-08-28T00:55:33.286ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166") failed: Property not found. retry in 1.023097847s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166
13802024-08-28T00:55:33.411ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf") failed: Property not found. retry in 1.4451509s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf
13812024-08-28T00:55:33.419ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def") failed: Property not found. retry in 1.382616611s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def
13822024-08-28T00:55:33.513ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f") failed: Property not found. retry in 823.753479ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f
13832024-08-28T00:55:33.614ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7") failed: Property not found. retry in 667.642402ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7
13842024-08-28T00:55:33.634ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff") failed: Property not found. retry in 1.172848098s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff
13852024-08-28T00:55:33.677ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba") failed: Property not found. retry in 672.862116ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba
13862024-08-28T00:55:33.694ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf") failed: Property not found. retry in 579.851369ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf
13872024-08-28T00:55:33.707ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e") failed: Property not found. retry in 867.90412ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e
13882024-08-28T00:55:33.720ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739") failed: Property not found. retry in 589.570888ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739
13892024-08-28T00:55:33.763ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd") failed: Property not found. retry in 525.983904ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd
13902024-08-28T00:55:33.899ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71") failed: Property not found. retry in 1.448552303s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71
13912024-08-28T00:55:33.937ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26") failed: Property not found. retry in 1.291168671s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26
13922024-08-28T00:55:34.032ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1") failed: Property not found. retry in 660.57262ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1
13932024-08-28T00:55:34.137ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7") failed: Property not found. retry in 814.961506ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7
13942024-08-28T00:55:34.205ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b") failed: Property not found. retry in 986.522493ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b
13952024-08-28T00:55:34.299ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf") failed: Property not found. retry in 1.483785575s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf
13962024-08-28T00:55:34.312ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517") failed: Property not found. retry in 1.011254844s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517
13972024-08-28T00:55:34.324ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd") failed: Property not found. retry in 1.303154344s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd
13982024-08-28T00:55:34.338ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7") failed: Property not found. retry in 842.815794ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7
13992024-08-28T00:55:34.347ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166") failed: Property not found. retry in 1.46231326s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166
14002024-08-28T00:55:34.356ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739") failed: Property not found. retry in 1.214842574s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739
14012024-08-28T00:55:34.363ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba") failed: Property not found. retry in 811.526518ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba
14022024-08-28T00:55:34.371ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f") failed: Property not found. retry in 701.736698ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f
14032024-08-28T00:55:34.471ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55") failed: Property not found. retry in 735.311703ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55
14042024-08-28T00:55:34.587ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e") failed: Property not found. retry in 829.495786ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e
14052024-08-28T00:55:34.596ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d") failed: Property not found. retry in 536.226921ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d
14062024-08-28T00:55:34.703ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1") failed: Property not found. retry in 1.126774147s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1
14072024-08-28T00:55:34.813ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def") failed: Property not found. retry in 1.335575514s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def
14082024-08-28T00:55:34.822ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff") failed: Property not found. retry in 1.354895555s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff
14092024-08-28T00:55:34.869ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf") failed: Property not found. retry in 1.437858921s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf
14102024-08-28T00:55:34.966ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7") failed: Property not found. retry in 969.078137ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7
14112024-08-28T00:55:35.084ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f") failed: Property not found. retry in 526.889166ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f
14122024-08-28T00:55:35.144ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d") failed: Property not found. retry in 1.390026203s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d
14132024-08-28T00:55:35.188ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba") failed: Property not found. retry in 1.150039441s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba
14142024-08-28T00:55:35.202ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7") failed: Property not found. retry in 1.43916251s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7
14152024-08-28T00:55:35.214ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b") failed: Property not found. retry in 646.724887ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b
14162024-08-28T00:55:35.224ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55") failed: Property not found. retry in 1.028436696s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55
14172024-08-28T00:55:35.239ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26") failed: Property not found. retry in 904.189762ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26
14182024-08-28T00:55:35.339ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517") failed: Property not found. retry in 1.487910167s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517
14192024-08-28T00:55:35.362ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71") failed: Property not found. retry in 991.42332ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71
14202024-08-28T00:55:35.430ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e") failed: Property not found. retry in 1.374079008s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e
14212024-08-28T00:55:35.587ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739") failed: Property not found. retry in 1.471683858s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739
14222024-08-28T00:55:35.624ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f") failed: Property not found. retry in 745.9488ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f
14232024-08-28T00:55:35.639ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd") failed: Property not found. retry in 611.78295ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd
14242024-08-28T00:55:35.795ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf") failed: Property not found. retry in 761.141089ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf
14252024-08-28T00:55:35.821ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166") failed: Property not found. retry in 1.467618314s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166
14262024-08-28T00:55:35.842ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1") failed: Property not found. retry in 1.084651686s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1
14272024-08-28T00:55:35.873ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b") failed: Property not found. retry in 866.967466ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b
14282024-08-28T00:55:35.946ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7") failed: Property not found. retry in 828.08537ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7
14292024-08-28T00:55:36.154ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26") failed: Property not found. retry in 1.126718907s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26
14302024-08-28T00:55:36.163ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def") failed: Property not found. retry in 1.474017229s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def
14312024-08-28T00:55:36.185ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff") failed: Property not found. retry in 540.433949ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff
14322024-08-28T00:55:36.261ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd") failed: Property not found. retry in 1.037277237s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd
14332024-08-28T00:55:36.271ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55") failed: Property not found. retry in 1.019134083s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55
14342024-08-28T00:55:36.316ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf") failed: Property not found. retry in 1.299489521s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf
14352024-08-28T00:55:36.350ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba") failed: Property not found. retry in 852.333068ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba
14362024-08-28T00:55:36.363ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71") failed: Property not found. retry in 1.130771596s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71
14372024-08-28T00:55:36.381ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f") failed: Property not found. retry in 726.656901ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f
14382024-08-28T00:55:36.546ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d") failed: Property not found. retry in 608.817767ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d
14392024-08-28T00:55:36.567ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf") failed: Property not found. retry in 952.663681ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf
14402024-08-28T00:55:36.652ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7") failed: Property not found. retry in 1.374127419s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7
14412024-08-28T00:55:36.736ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff") failed: Property not found. retry in 1.198221289s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff
14422024-08-28T00:55:36.754ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b") failed: Property not found. retry in 926.78306ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b
14432024-08-28T00:55:36.786ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7") failed: Property not found. retry in 781.692141ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7
14442024-08-28T00:55:36.817ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e") failed: Property not found. retry in 1.06758194s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e
14452024-08-28T00:55:36.838ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517") failed: Property not found. retry in 845.738824ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517
14462024-08-28T00:55:36.941ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1") failed: Property not found. retry in 753.152715ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1
14472024-08-28T00:55:37.070ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739") failed: Property not found. retry in 820.72169ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739
14482024-08-28T00:55:37.120ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f") failed: Property not found. retry in 1.120929417s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f
14492024-08-28T00:55:37.168ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d") failed: Property not found. retry in 1.408786169s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d
14502024-08-28T00:55:37.215ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba") failed: Property not found. retry in 1.444316804s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba
14512024-08-28T00:55:37.293ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26") failed: Property not found. retry in 1.322935097s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26
14522024-08-28T00:55:37.306ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166") failed: Property not found. retry in 858.43844ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166
14532024-08-28T00:55:37.319ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55") failed: Property not found. retry in 1.391608769s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55
14542024-08-28T00:55:37.333ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd") failed: Property not found. retry in 1.474505146s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd
14552024-08-28T00:55:37.506ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71") failed: Property not found. retry in 1.198262606s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71
14562024-08-28T00:55:37.532ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf") failed: Property not found. retry in 1.056012676s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf
14572024-08-28T00:55:37.580ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7") failed: Property not found. retry in 687.62041ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7
14582024-08-28T00:55:37.627ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf") failed: Property not found. retry in 921.325542ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf
14592024-08-28T00:55:37.648ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def") failed: Property not found. retry in 1.170444052s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def
14602024-08-28T00:55:37.692ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b") failed: Property not found. retry in 908.122618ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b
14612024-08-28T00:55:37.703ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517") failed: Property not found. retry in 1.452803403s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517
14622024-08-28T00:55:37.713ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1") failed: Property not found. retry in 535.090876ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1
14632024-08-28T00:55:37.895ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e") failed: Property not found. retry in 559.060267ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e
14642024-08-28T00:55:37.905ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739") failed: Property not found. retry in 502.36852ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739
14652024-08-28T00:55:37.945ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff") failed: Property not found. retry in 1.375445816s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff
14662024-08-28T00:55:38.041ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7") failed: Property not found. retry in 1.422392555s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7
14672024-08-28T00:55:38.184ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166") failed: Property not found. retry in 1.109222533s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166
14682024-08-28T00:55:38.255ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f") failed: Property not found. retry in 1.49704288s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f
14692024-08-28T00:55:38.267ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1") failed: Property not found. retry in 530.543288ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1
14702024-08-28T00:55:38.281ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7") failed: Property not found. retry in 808.434494ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7
14712024-08-28T00:55:38.426ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739") failed: Property not found. retry in 756.700017ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739
14722024-08-28T00:55:38.481ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e") failed: Property not found. retry in 933.175308ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e
14732024-08-28T00:55:38.566ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf") failed: Property not found. retry in 1.001894667s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf
14742024-08-28T00:55:38.598ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d") failed: Property not found. retry in 1.048329721s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d
14752024-08-28T00:55:38.617ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf") failed: Property not found. retry in 638.500599ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf
14762024-08-28T00:55:38.638ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b") failed: Property not found. retry in 550.206045ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b
14772024-08-28T00:55:38.658ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26") failed: Property not found. retry in 1.130163211s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26
14782024-08-28T00:55:38.681ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba") failed: Property not found. retry in 668.534458ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba
14792024-08-28T00:55:38.726ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71") failed: Property not found. retry in 1.482501198s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71
14802024-08-28T00:55:38.743ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55") failed: Property not found. retry in 1.028238207s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55
14812024-08-28T00:55:38.819ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1") failed: Property not found. retry in 946.074901ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1
14822024-08-28T00:55:38.840ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd") failed: Property not found. retry in 1.450863449s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd
14832024-08-28T00:55:38.862ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def") failed: Property not found. retry in 1.239010586s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def
14842024-08-28T00:55:39.112ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7") failed: Property not found. retry in 1.078943325s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7
14852024-08-28T00:55:39.180ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517") failed: Property not found. retry in 1.418686469s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517
14862024-08-28T00:55:39.205ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739") failed: Property not found. retry in 1.139544411s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739
14872024-08-28T00:55:39.224ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b") failed: Property not found. retry in 984.932291ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b
14882024-08-28T00:55:39.280ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf") failed: Property not found. retry in 942.302301ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf
14892024-08-28T00:55:39.335ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166") failed: Property not found. retry in 1.356183044s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166
14902024-08-28T00:55:39.360ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff") failed: Property not found. retry in 1.437407473s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff
14912024-08-28T00:55:39.382ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba") failed: Property not found. retry in 1.361250462s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba
14922024-08-28T00:55:39.434ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e") failed: Property not found. retry in 1.142409411s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e
14932024-08-28T00:55:39.483ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7") failed: Property not found. retry in 1.067328999s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7
14942024-08-28T00:55:39.589ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf") failed: Property not found. retry in 1.001289135s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf
14952024-08-28T00:55:39.662ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d") failed: Property not found. retry in 789.697452ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d
14962024-08-28T00:55:39.770ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f") failed: Property not found. retry in 1.40951876s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f
14972024-08-28T00:55:39.789ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1") failed: Property not found. retry in 1.351659401s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1
14982024-08-28T00:55:39.805ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26") failed: Property not found. retry in 783.63396ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26
14992024-08-28T00:55:39.821ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55") failed: Property not found. retry in 1.102378941s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55
15002024-08-28T00:55:40.117ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def") failed: Property not found. retry in 1.086975636s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def
15012024-08-28T00:55:40.207ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7") failed: Property not found. retry in 1.006356337s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7
15022024-08-28T00:55:40.225ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71") failed: Property not found. retry in 1.013375s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71
15032024-08-28T00:55:40.236ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf") failed: Property not found. retry in 631.054654ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf
15042024-08-28T00:55:40.251ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b") failed: Property not found. retry in 578.564947ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b
15052024-08-28T00:55:40.309ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd") failed: Property not found. retry in 1.065247201s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd
15062024-08-28T00:55:40.361ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739") failed: Property not found. retry in 550.39911ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739
15072024-08-28T00:55:40.467ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d") failed: Property not found. retry in 1.341408506s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d
15082024-08-28T00:55:40.566ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7") failed: Property not found. retry in 515.315964ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7
15092024-08-28T00:55:40.595ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e") failed: Property not found. retry in 889.437808ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e
15102024-08-28T00:55:40.608ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf") failed: Property not found. retry in 966.151345ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf
15112024-08-28T00:55:40.625ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26") failed: Property not found. retry in 780.487393ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26
15122024-08-28T00:55:40.639ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517") failed: Property not found. retry in 1.115269579s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517
15132024-08-28T00:55:40.711ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166") failed: Property not found. retry in 956.153394ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166
15142024-08-28T00:55:40.759ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba") failed: Property not found. retry in 1.184007824s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba
15152024-08-28T00:55:40.816ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff") failed: Property not found. retry in 1.337439919s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff
15162024-08-28T00:55:40.850ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b") failed: Property not found. retry in 1.350061378s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b
15172024-08-28T00:55:40.883ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf") failed: Property not found. retry in 1.208744956s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf
15182024-08-28T00:55:40.927ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739") failed: Property not found. retry in 1.409679883s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739
15192024-08-28T00:55:40.939ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55") failed: Property not found. retry in 877.38971ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55
15202024-08-28T00:55:41.094ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7") failed: Property not found. retry in 1.037790073s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7
15212024-08-28T00:55:41.151ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1") failed: Property not found. retry in 880.993002ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1
15222024-08-28T00:55:41.191ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f") failed: Property not found. retry in 1.480180021s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f
15232024-08-28T00:55:41.218ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def") failed: Property not found. retry in 1.009721552s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def
15242024-08-28T00:55:41.229ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7") failed: Property not found. retry in 1.496311771s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7
15252024-08-28T00:55:41.250ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71") failed: Property not found. retry in 1.303317381s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71
15262024-08-28T00:55:41.386ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd") failed: Property not found. retry in 1.222851097s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd
15272024-08-28T00:55:41.498ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e") failed: Property not found. retry in 1.265423393s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e
15282024-08-28T00:55:41.589ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf") failed: Property not found. retry in 859.299532ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf
15292024-08-28T00:55:41.679ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166") failed: Property not found. retry in 627.175075ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166
15302024-08-28T00:55:41.765ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517") failed: Property not found. retry in 932.209763ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517
15312024-08-28T00:55:41.818ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d") failed: Property not found. retry in 1.114516747s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d
15322024-08-28T00:55:41.931ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 32.785058702s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
15332024-08-28T00:55:41.955ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba") failed: Property not found. retry in 984.631849ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba
15342024-08-28T00:55:42.044ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1") failed: Property not found. retry in 592.891216ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1
15352024-08-28T00:55:42.102ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf") failed: Property not found. retry in 1.317005039s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf
15362024-08-28T00:55:42.143ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7") failed: Property not found. retry in 1.354186461s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7
15372024-08-28T00:55:42.163ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff") failed: Property not found. retry in 1.024987539s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff
15382024-08-28T00:55:42.319ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166") failed: Property not found. retry in 947.599277ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166
15392024-08-28T00:55:42.355ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739") failed: Property not found. retry in 1.340157339s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739
15402024-08-28T00:55:42.577ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71") failed: Property not found. retry in 1.232955891s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71
15412024-08-28T00:55:42.980ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba") failed: Property not found. retry in 604.239359ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba
15422024-08-28T00:55:43.603ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba") failed: Property not found. retry in 1.473571679s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba
15432024-08-28T00:55:45.100ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba") failed: Property not found. retry in 531.533557ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba
15442024-08-28T00:55:45.646ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba") failed: Property not found. retry in 547.886673ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba
15452024-08-28T00:55:46.207ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba") failed: Property not found. retry in 632.635753ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba
15462024-08-28T00:55:46.850ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba") failed: Property not found. retry in 766.923134ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba
15472024-08-28T00:55:47.629ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba") failed: Property not found. retry in 696.528694ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba
15482024-08-28T00:55:48.336ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba") failed: Property not found. retry in 853.211044ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba
15492024-08-28T00:55:48.918ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/45733be1-aef6-43c8-be71-719893a1feda): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/nexus.rs:571 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
15502024-08-28T00:55:49.200ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba") failed: Property not found. retry in 1.294931623s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba
15512024-08-28T00:55:50.291ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/45733be1-aef6-43c8-be71-719893a1feda): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/nexus.rs:571 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
15522024-08-28T00:55:50.505ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba") failed: Property not found. retry in 1.333099642s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba
15532024-08-28T00:55:51.849ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba") failed: Property not found. retry in 1.089852685s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba
15542024-08-28T00:55:52.176ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/45733be1-aef6-43c8-be71-719893a1feda): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/probe_manager.rs:230 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
15552024-08-28T00:55:52.291ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/45733be1-aef6-43c8-be71-719893a1feda): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/nexus.rs:571 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
15562024-08-28T00:55:52.967ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_nexus_4ee19452-709a-4abd-a238-7af80f61d166
15572024-08-28T00:55:52.968ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_nexus_a5227819-155d-4a08-8e71-bd8a327ad739
15582024-08-28T00:55:52.968ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_clickhouse_b5e20b06-936c-44f5-a47b-458ef7fd56ba
15592024-08-28T00:55:52.968ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_crucible_pantry_2ddb8173-d078-4b99-bac5-ed06b4cf9def
15602024-08-28T00:55:52.968ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_external_dns_fa9e7556-b41e-4114-bf3f-e918820a9bcf
15612024-08-28T00:55:52.968ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_crucible_pantry_dbf1e583-6b72-47ae-8a84-3ce2331a120b
15622024-08-28T00:55:52.968ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_crucible_3d63d074-b018-4e87-8045-b03641d9d1f1
15632024-08-28T00:55:52.968ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_crucible_02dc2ce9-7b01-425e-b378-1a4d032273cf
15642024-08-28T00:55:52.968ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_oximeter_4fe6d00e-187b-4dd4-9562-e20645ddec26
15652024-08-28T00:55:52.968ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_crucible_15a66369-82a9-40a2-a682-fbeb107fc5d7
15662024-08-28T00:55:52.968ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_external_dns_2cae9f85-8043-498f-af8e-a4bc9724ac55
15672024-08-28T00:55:52.968ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_crucible_d14a7041-b90b-487e-acaa-0c8f6daa795f
15682024-08-28T00:55:52.968ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_crucible_0ed664b9-9a6f-41dc-a5f5-b38df3412517
15692024-08-28T00:55:52.968ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_crucible_pantry_8a6d7ab2-57bd-4341-9872-076f7f2d8bfd
15702024-08-28T00:55:52.968ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_crucible_c3f1e562-0d21-4f0e-863a-66d20b9899f7
15712024-08-28T00:55:52.968ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_crucible_68525002-65a3-43fc-8e3f-0c38a0d081ff
15722024-08-28T00:55:52.968ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_crucible_bed84857-bc98-412a-90cd-2b38b7020c71
15732024-08-28T00:55:52.968ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_crucible_36357f09-d97f-4b2f-a4fd-16007d12067e
15742024-08-28T00:55:52.968ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_nexus_f840e071-026c-488d-b1c9-2c4d1ce3e74d
15752024-08-28T00:55:52.968ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/1d8f47b2-ceb5-4768-b5e0-de73f9c6288a/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
15762024-08-28T00:55:52.970ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/2701509e-5560-4f63-bb12-b820fcc5e4cb/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
15772024-08-28T00:55:52.978ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 41466897 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:39426 req_id = fe30cbfb-8786-4104-bb87-819cac841129 response_code = 204 uri = /omicron-zones
15782024-08-28T00:55:52.985ZINFOSledAgent (RSS): Finished setting up services file = sled-agent/src/rack_setup/service.rs:1286
15792024-08-28T00:55:52.985ZINFOSledAgent (RSS): Writing ledger to /pool/int/1d8f47b2-ceb5-4768-b5e0-de73f9c6288a/config/.rss-plan-completed.marker.tmp file = common/src/ledger.rs:199
15802024-08-28T00:55:52.985ZINFOSledAgent (RSS): Writing ledger to /pool/int/2701509e-5560-4f63-bb12-b820fcc5e4cb/config/.rss-plan-completed.marker.tmp file = common/src/ledger.rs:199
15812024-08-28T00:55:52.988ZINFOSledAgent (RSS): Handing off control to Nexus file = sled-agent/src/rack_setup/service.rs:687
15822024-08-28T00:55:52.990ZINFOSledAgent (RSS): Nexus address: [fd00:1122:3344:101::c]:12221 file = sled-agent/src/rack_setup/service.rs:700
15832024-08-28T00:55:53.027ZINFOSledAgent (RSS): rack_network_config: RackNetworkConfigV2 { bfd: [], bgp: [], infra_ip_first: 10.151.1.174, infra_ip_last: 10.151.1.174, ports: [ PortConfigV2 { addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.1.174, width: 24, }, ), vlan_id: None, }, ], autoneg: false, bgp_peers: [], lldp: None, port: "qsfp0", routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), local_pref: None, nexthop: 10.151.1.100, vlan_id: None, }, ], switch: Switch0, uplink_port_fec: None, uplink_port_speed: Speed40G, }, ], rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, } file = sled-agent/src/rack_setup/service.rs:867
15842024-08-28T00:55:53.030ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/75a88ee6-61ab-40f4-a8f1-caf11e1c9852/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
15852024-08-28T00:55:53.105ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/75a88ee6-61ab-40f4-a8f1-caf11e1c9852/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
15862024-08-28T00:55:53.184ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/45733be1-aef6-43c8-be71-719893a1feda): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/probe_manager.rs:230 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
15872024-08-28T00:55:53.221ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/75a88ee6-61ab-40f4-a8f1-caf11e1c9852/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
15882024-08-28T00:55:53.343ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/75a88ee6-61ab-40f4-a8f1-caf11e1c9852/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
15892024-08-28T00:55:53.467ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/75a88ee6-61ab-40f4-a8f1-caf11e1c9852/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
15902024-08-28T00:55:53.635ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/75a88ee6-61ab-40f4-a8f1-caf11e1c9852/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
15912024-08-28T00:55:53.903ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/75a88ee6-61ab-40f4-a8f1-caf11e1c9852/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
15922024-08-28T00:55:54.153ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/75a88ee6-61ab-40f4-a8f1-caf11e1c9852/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
15932024-08-28T00:55:54.186ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/45733be1-aef6-43c8-be71-719893a1feda): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/probe_manager.rs:230 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
15942024-08-28T00:55:54.292ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/45733be1-aef6-43c8-be71-719893a1feda): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/nexus.rs:571 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
15952024-08-28T00:55:54.372ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:62499
15962024-08-28T00:55:54.372ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 33 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:62499 req_id = 5031eca5-5443-4879-83eb-d9131ad77877 response_code = 200 uri = /baseboard
15972024-08-28T00:55:54.549ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/75a88ee6-61ab-40f4-a8f1-caf11e1c9852/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
15982024-08-28T00:55:55.077ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/75a88ee6-61ab-40f4-a8f1-caf11e1c9852/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
15992024-08-28T00:55:55.189ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/45733be1-aef6-43c8-be71-719893a1feda): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/probe_manager.rs:230 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
16002024-08-28T00:55:55.591ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/75a88ee6-61ab-40f4-a8f1-caf11e1c9852/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
16012024-08-28T00:55:55.963ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/75a88ee6-61ab-40f4-a8f1-caf11e1c9852/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
16022024-08-28T00:55:56.193ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/45733be1-aef6-43c8-be71-719893a1feda): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/probe_manager.rs:230 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
16032024-08-28T00:55:56.290ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/45733be1-aef6-43c8-be71-719893a1feda): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/nexus.rs:571 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
16042024-08-28T00:55:56.562ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/75a88ee6-61ab-40f4-a8f1-caf11e1c9852/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
16052024-08-28T00:55:57.098ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/75a88ee6-61ab-40f4-a8f1-caf11e1c9852/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
16062024-08-28T00:55:57.197ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/45733be1-aef6-43c8-be71-719893a1feda): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/probe_manager.rs:230 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
16072024-08-28T00:55:57.994ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/75a88ee6-61ab-40f4-a8f1-caf11e1c9852/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
16082024-08-28T00:55:58.202ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/45733be1-aef6-43c8-be71-719893a1feda): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/probe_manager.rs:230 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
16092024-08-28T00:55:58.291ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/45733be1-aef6-43c8-be71-719893a1feda): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/nexus.rs:571 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
16102024-08-28T00:55:59.204ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/45733be1-aef6-43c8-be71-719893a1feda): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/probe_manager.rs:230 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
16112024-08-28T00:55:59.825ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/75a88ee6-61ab-40f4-a8f1-caf11e1c9852/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
16122024-08-28T00:56:00.206ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/45733be1-aef6-43c8-be71-719893a1feda): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/probe_manager.rs:230 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
16132024-08-28T00:56:00.291ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/45733be1-aef6-43c8-be71-719893a1feda): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/nexus.rs:571 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
16142024-08-28T00:56:01.210ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/45733be1-aef6-43c8-be71-719893a1feda): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/probe_manager.rs:230 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
16152024-08-28T00:56:01.423ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/75a88ee6-61ab-40f4-a8f1-caf11e1c9852/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
16162024-08-28T00:56:02.213ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/45733be1-aef6-43c8-be71-719893a1feda): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/probe_manager.rs:230 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
16172024-08-28T00:56:02.291ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/45733be1-aef6-43c8-be71-719893a1feda): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/nexus.rs:571 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
16182024-08-28T00:56:03.059ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/75a88ee6-61ab-40f4-a8f1-caf11e1c9852/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
16192024-08-28T00:56:03.217ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/45733be1-aef6-43c8-be71-719893a1feda): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/probe_manager.rs:230 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
16202024-08-28T00:56:04.211ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/75a88ee6-61ab-40f4-a8f1-caf11e1c9852/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
16212024-08-28T00:56:04.220ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/45733be1-aef6-43c8-be71-719893a1feda): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/probe_manager.rs:230 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
16222024-08-28T00:56:04.291ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/45733be1-aef6-43c8-be71-719893a1feda): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/nexus.rs:571 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
16232024-08-28T00:56:08.120ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "7e4eec5b-85dd-4bc4-88bf-ce17c670c947", "content-length": "173", "date": "Wed, 28 Aug 2024 00:56:07 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"45733be1-aef6-43c8-be71-719893a1feda\"", request_id: "7e4eec5b-85dd-4bc4-88bf-ce17c670c947" } file = sled-agent/src/rack_setup/service.rs:929
16242024-08-28T00:56:08.378ZINFOSledAgent (NexusNotifierTask): Successfully put SledAgentInfo to nexus file = sled-agent/src/nexus.rs:544 new_generation = 1 old_generation = 1 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
16252024-08-28T00:56:08.520ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:42085
16262024-08-28T00:56:08.520ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
16272024-08-28T00:56:08.618ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 97791 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:42085 req_id = 2683fdc7-9763-4c0b-90a3-34e93a060e31 response_code = 200 uri = /inventory
16282024-08-28T00:56:08.618ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
16292024-08-28T00:56:08.618ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/1d8f47b2-ceb5-4768-b5e0-de73f9c6288a/config/omicron-zones.json file = common/src/ledger.rs:177
16302024-08-28T00:56:08.620ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/2701509e-5560-4f63-bb12-b820fcc5e4cb/config/omicron-zones.json file = common/src/ledger.rs:177
16312024-08-28T00:56:08.620ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1641 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:42085 req_id = 504dd357-7789-448e-bb5f-c0a315994355 response_code = 200 uri = /omicron-zones
16322024-08-28T00:56:13.644ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:49570
16332024-08-28T00:56:13.645ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 1eb980bf-46dc-45a7-aa4f-098bf58d6904, kind: Service { id: 2cae9f85-8043-498f-af8e-a4bc9724ac55 }, name: Name("external-dns-2cae9f85-8043-498f-af8e-a4bc9724ac55"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 212, 38])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: eafd744b-ffb5-445c-8366-bd33299374a6, kind: Service { id: fa9e7556-b41e-4114-bf3f-e918820a9bcf }, name: Name("external-dns-fa9e7556-b41e-4114-bf3f-e918820a9bcf"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 202, 132])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 16870a00-56de-4335-bbe4-e10f8544f208, kind: Service { id: f840e071-026c-488d-b1c9-2c4d1ce3e74d }, name: Name("nexus-f840e071-026c-488d-b1c9-2c4d1ce3e74d"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 209, 34])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 707af25c-dc9c-42b2-9351-0dc3824f26c9, kind: Service { id: a5227819-155d-4a08-8e71-bd8a327ad739 }, name: Name("nexus-a5227819-155d-4a08-8e71-bd8a327ad739"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 220, 190])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a6310629-11e4-4b6d-b328-1d8767a4ab69, kind: Service { id: 4ee19452-709a-4abd-a238-7af80f61d166 }, name: Name("nexus-4ee19452-709a-4abd-a238-7af80f61d166"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 191, 68])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
16342024-08-28T00:56:13.645ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
16352024-08-28T00:56:13.645ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
16362024-08-28T00:56:13.645ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
16372024-08-28T00:56:13.645ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
16382024-08-28T00:56:13.645ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
16392024-08-28T00:56:13.645ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
16402024-08-28T00:56:13.645ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 817 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:49570 req_id = e58cbd28-c807-4faf-b943-df42d9bff6db response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
16412024-08-28T00:56:13.646ZINFOSledAgent (RSS): Handoff to Nexus is complete file = sled-agent/src/rack_setup/service.rs:939
16422024-08-28T00:56:13.930ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:45955
16432024-08-28T00:56:13.930ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
16442024-08-28T00:56:14.025ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 94790 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:45955 req_id = 759f2531-75b1-49b0-a115-216d53f2f1da response_code = 200 uri = /inventory
16452024-08-28T00:56:14.025ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
16462024-08-28T00:56:14.025ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/1d8f47b2-ceb5-4768-b5e0-de73f9c6288a/config/omicron-zones.json file = common/src/ledger.rs:177
16472024-08-28T00:56:14.026ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/2701509e-5560-4f63-bb12-b820fcc5e4cb/config/omicron-zones.json file = common/src/ledger.rs:177
16482024-08-28T00:56:14.026ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 654 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:45955 req_id = 190c0e69-e52d-472a-b639-3eadf9048a4f response_code = 200 uri = /omicron-zones
16492024-08-28T00:56:14.755ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 123.954800229s error = "Error Response: status: 503 Service Unavailable; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"26e2e699-aa3c-4a97-b5c0-cc475e12e6eb\\", \\"content-length\\": \\"133\\", \\"date\\": \\"Wed, 28 Aug 2024 00:56:14 GMT\\"}; value: Error { error_code: Some(\\"ServiceNotAvailable\\"), message: \\"Service Unavailable\\", request_id: \\"26e2e699-aa3c-4a97-b5c0-cc475e12e6eb\\" }" file = oximeter/producer/src/lib.rs:424 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
16502024-08-28T00:56:14.980ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:64620
16512024-08-28T00:56:14.980ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 1eb980bf-46dc-45a7-aa4f-098bf58d6904, kind: Service { id: 2cae9f85-8043-498f-af8e-a4bc9724ac55 }, name: Name("external-dns-2cae9f85-8043-498f-af8e-a4bc9724ac55"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 212, 38])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: eafd744b-ffb5-445c-8366-bd33299374a6, kind: Service { id: fa9e7556-b41e-4114-bf3f-e918820a9bcf }, name: Name("external-dns-fa9e7556-b41e-4114-bf3f-e918820a9bcf"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 202, 132])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 16870a00-56de-4335-bbe4-e10f8544f208, kind: Service { id: f840e071-026c-488d-b1c9-2c4d1ce3e74d }, name: Name("nexus-f840e071-026c-488d-b1c9-2c4d1ce3e74d"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 209, 34])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 707af25c-dc9c-42b2-9351-0dc3824f26c9, kind: Service { id: a5227819-155d-4a08-8e71-bd8a327ad739 }, name: Name("nexus-a5227819-155d-4a08-8e71-bd8a327ad739"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 220, 190])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a6310629-11e4-4b6d-b328-1d8767a4ab69, kind: Service { id: 4ee19452-709a-4abd-a238-7af80f61d166 }, name: Name("nexus-4ee19452-709a-4abd-a238-7af80f61d166"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 191, 68])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
16522024-08-28T00:56:14.980ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
16532024-08-28T00:56:14.980ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
16542024-08-28T00:56:14.980ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
16552024-08-28T00:56:14.980ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
16562024-08-28T00:56:14.980ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
16572024-08-28T00:56:14.980ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
16582024-08-28T00:56:14.980ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 250 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:64620 req_id = 3ddea7a1-15b6-41c9-88a9-5cc2ef48680c response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
16592024-08-28T00:56:15.310ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:51593
16602024-08-28T00:56:15.310ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 1eb980bf-46dc-45a7-aa4f-098bf58d6904, kind: Service { id: 2cae9f85-8043-498f-af8e-a4bc9724ac55 }, name: Name("external-dns-2cae9f85-8043-498f-af8e-a4bc9724ac55"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 212, 38])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: eafd744b-ffb5-445c-8366-bd33299374a6, kind: Service { id: fa9e7556-b41e-4114-bf3f-e918820a9bcf }, name: Name("external-dns-fa9e7556-b41e-4114-bf3f-e918820a9bcf"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 202, 132])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 16870a00-56de-4335-bbe4-e10f8544f208, kind: Service { id: f840e071-026c-488d-b1c9-2c4d1ce3e74d }, name: Name("nexus-f840e071-026c-488d-b1c9-2c4d1ce3e74d"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 209, 34])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 707af25c-dc9c-42b2-9351-0dc3824f26c9, kind: Service { id: a5227819-155d-4a08-8e71-bd8a327ad739 }, name: Name("nexus-a5227819-155d-4a08-8e71-bd8a327ad739"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 220, 190])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a6310629-11e4-4b6d-b328-1d8767a4ab69, kind: Service { id: 4ee19452-709a-4abd-a238-7af80f61d166 }, name: Name("nexus-4ee19452-709a-4abd-a238-7af80f61d166"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 191, 68])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
16612024-08-28T00:56:15.310ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
16622024-08-28T00:56:15.310ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
16632024-08-28T00:56:15.310ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
16642024-08-28T00:56:15.310ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
16652024-08-28T00:56:15.310ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
16662024-08-28T00:56:15.310ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
16672024-08-28T00:56:15.310ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 229 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:51593 req_id = c4f6a32b-7d8e-4fb6-9af5-9254e17131ee response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
16682024-08-28T00:56:15.352ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:59652
16692024-08-28T00:56:15.352ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 1eb980bf-46dc-45a7-aa4f-098bf58d6904, kind: Service { id: 2cae9f85-8043-498f-af8e-a4bc9724ac55 }, name: Name("external-dns-2cae9f85-8043-498f-af8e-a4bc9724ac55"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 212, 38])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: eafd744b-ffb5-445c-8366-bd33299374a6, kind: Service { id: fa9e7556-b41e-4114-bf3f-e918820a9bcf }, name: Name("external-dns-fa9e7556-b41e-4114-bf3f-e918820a9bcf"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 202, 132])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 16870a00-56de-4335-bbe4-e10f8544f208, kind: Service { id: f840e071-026c-488d-b1c9-2c4d1ce3e74d }, name: Name("nexus-f840e071-026c-488d-b1c9-2c4d1ce3e74d"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 209, 34])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 707af25c-dc9c-42b2-9351-0dc3824f26c9, kind: Service { id: a5227819-155d-4a08-8e71-bd8a327ad739 }, name: Name("nexus-a5227819-155d-4a08-8e71-bd8a327ad739"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 220, 190])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a6310629-11e4-4b6d-b328-1d8767a4ab69, kind: Service { id: 4ee19452-709a-4abd-a238-7af80f61d166 }, name: Name("nexus-4ee19452-709a-4abd-a238-7af80f61d166"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 191, 68])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
16702024-08-28T00:56:15.352ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
16712024-08-28T00:56:15.352ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
16722024-08-28T00:56:15.352ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
16732024-08-28T00:56:15.352ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
16742024-08-28T00:56:15.352ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
16752024-08-28T00:56:15.352ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
16762024-08-28T00:56:15.352ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 160 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:59652 req_id = b0344a83-4c2f-4f40-82c3-9bf0e658291c response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
16772024-08-28T00:56:19.699ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:43740
16782024-08-28T00:56:19.699ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 68 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:43740 req_id = e716ff2f-10d1-46be-9847-31749155280e response_code = 200 uri = /vpc-routes
16792024-08-28T00:56:19.773ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2297 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:43740 req_id = 43cc3009-4ff0-4b35-aef1-7be3bd048c86 response_code = 204 uri = /vpc-routes
16802024-08-28T00:56:24.406ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:46517
16812024-08-28T00:56:24.406ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 51 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:46517 req_id = db5f7715-fb8a-4175-a5ef-a9f7770d9fe2 response_code = 200 uri = /baseboard
16822024-08-28T00:56:36.352ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:45542
16832024-08-28T00:56:36.352ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 85 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:45542 req_id = e6df7017-e98a-48a3-b133-a4acd4d8442f response_code = 200 uri = /vpc-routes
16842024-08-28T00:56:36.397ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:42035
16852024-08-28T00:56:36.397ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 159 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:42035 req_id = 6b991feb-0043-4d55-9166-5024fa4d7445 response_code = 200 uri = /v2p
16862024-08-28T00:56:36.407ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 105 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:45542 req_id = 8e6a1e02-8e10-4a0e-ac4c-98e7c92f92e9 response_code = 204 uri = /vpc-routes
16872024-08-28T00:56:36.540ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:54444
16882024-08-28T00:56:36.540ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
16892024-08-28T00:56:36.603ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
16902024-08-28T00:56:36.664ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 123916 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:54444 req_id = 5c59afbd-0c6b-4e03-b7db-8bd4c528767c response_code = 204 uri = /switch-ports
16912024-08-28T00:56:36.665ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 990 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:54444 req_id = da9ed296-5b7a-426a-9fe5-4544ea6d24ab response_code = 200 uri = /network-bootstore-config
16922024-08-28T00:56:36.926ZINFOSledAgent (bootstore): Attempting to update network config with generation: 2, current_generation: 1 file = bootstore/src/schemes/v0/peer.rs:564 peer_id = pc-buskin-i86pc
16932024-08-28T00:56:36.926ZINFOSledAgent (bootstore): Writing ledger to /pool/int/1d8f47b2-ceb5-4768-b5e0-de73f9c6288a/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-buskin-i86pc
16942024-08-28T00:56:36.927ZINFOSledAgent (bootstore): Writing ledger to /pool/int/2701509e-5560-4f63-bb12-b820fcc5e4cb/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-buskin-i86pc
16952024-08-28T00:56:36.928ZINFOSledAgent (bootstore): Broadcasting network config with generation 2 file = bootstore/src/schemes/v0/peer.rs:648 peer_id = pc-buskin-i86pc
16962024-08-28T00:56:36.928ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2906 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:54444 req_id = 698165fb-e51e-4159-9078-f9baf178c2c5 response_code = 204 uri = /network-bootstore-config
16972024-08-28T00:56:37.028ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:39389
16982024-08-28T00:56:37.028ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 45 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:39389 req_id = 381cddff-a045-42f9-893c-4dca2ceaf92b response_code = 200 uri = /vpc-routes
16992024-08-28T00:56:37.042ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 123 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:39389 req_id = 02911e47-552d-463c-8b59-016c6393b938 response_code = 204 uri = /vpc-routes
17002024-08-28T00:56:37.072ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:52256
17012024-08-28T00:56:37.072ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 109 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:52256 req_id = 7312c9c1-6547-4fa1-95ab-313d1f157132 response_code = 200 uri = /vpc-routes
17022024-08-28T00:56:37.092ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 133 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:52256 req_id = 71528c84-f4ce-49a9-a9e5-afacbd289639 response_code = 204 uri = /vpc-routes
17032024-08-28T00:56:37.163ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:49326
17042024-08-28T00:56:37.163ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
17052024-08-28T00:56:37.210ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
17062024-08-28T00:56:37.271ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 108459 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:49326 req_id = b02f0814-9617-4460-bb14-78fdeecd003d response_code = 204 uri = /switch-ports
17072024-08-28T00:56:37.271ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:61770
17082024-08-28T00:56:37.271ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:46120
17092024-08-28T00:56:37.272ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
17102024-08-28T00:56:37.272ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 782 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:61770 req_id = ff738483-d5cd-4aff-90c1-ac0cca79dfd5 response_code = 200 uri = /v2p
17112024-08-28T00:56:37.272ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 953 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:49326 req_id = 8bc883d2-f81a-451d-a81f-e993bc4baa0b response_code = 200 uri = /network-bootstore-config
17122024-08-28T00:56:37.303ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:48919
17132024-08-28T00:56:37.303ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 264 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:48919 req_id = 3edb886b-ceb6-45f6-b02e-19c825242623 response_code = 200 uri = /v2p
17142024-08-28T00:56:37.311ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
17152024-08-28T00:56:37.367ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 95200 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:46120 req_id = b9d4bf7b-9f5b-42d8-a350-0ff2f64bd234 response_code = 204 uri = /switch-ports
17162024-08-28T00:56:37.368ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 962 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:46120 req_id = 0f35310c-dae9-4d37-87c1-94f5d2e70ed7 response_code = 200 uri = /network-bootstore-config
17172024-08-28T00:56:44.051ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
17182024-08-28T00:56:54.445ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:64804
17192024-08-28T00:56:54.445ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 84 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:64804 req_id = b623947b-3e2b-4d14-8e36-9ef61e2c2066 response_code = 200 uri = /baseboard
17202024-08-28T00:57:06.262ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:43512
17212024-08-28T00:57:06.262ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 93 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:43512 req_id = 0e113438-b1f3-440c-a585-6aaa89e8fbf0 response_code = 200 uri = /vpc-routes
17222024-08-28T00:57:06.276ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 187 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:43512 req_id = 61509de9-4d3a-40b0-b24c-c2f7a87c76c3 response_code = 204 uri = /vpc-routes
17232024-08-28T00:57:06.614ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:40330
17242024-08-28T00:57:06.614ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 137 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:40330 req_id = 7979a3e0-7704-4b2c-bfbe-8a801dc86db5 response_code = 200 uri = /v2p
17252024-08-28T00:57:06.710ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:52523
17262024-08-28T00:57:06.710ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
17272024-08-28T00:57:06.749ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
17282024-08-28T00:57:06.806ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 96617 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:52523 req_id = 11311805-2718-4a74-bced-931dd8bb001a response_code = 204 uri = /switch-ports
17292024-08-28T00:57:06.808ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1409 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:52523 req_id = 18208c23-8134-48a7-aa33-7419e20b1ad6 response_code = 200 uri = /network-bootstore-config
17302024-08-28T00:57:07.086ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:43895
17312024-08-28T00:57:07.087ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 49 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:43895 req_id = 7decb17a-0545-4a6f-a84c-2906ade67584 response_code = 200 uri = /vpc-routes
17322024-08-28T00:57:07.102ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 131 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:43895 req_id = 77625cb7-9c74-4eb4-8d43-a250df1fb42c response_code = 204 uri = /vpc-routes
17332024-08-28T00:57:07.167ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:53153
17342024-08-28T00:57:07.167ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 44 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:53153 req_id = 5623547d-e722-4363-b824-e40a93c6851e response_code = 200 uri = /vpc-routes
17352024-08-28T00:57:07.180ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 119 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:53153 req_id = b7a51a81-534d-4dd1-9aee-67978ff02888 response_code = 204 uri = /vpc-routes
17362024-08-28T00:57:07.310ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:53266
17372024-08-28T00:57:07.310ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 165 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:53266 req_id = e40c9b6c-ed9c-4fb5-8c31-562fa6898cb7 response_code = 200 uri = /v2p
17382024-08-28T00:57:07.435ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:39247
17392024-08-28T00:57:07.435ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 155 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:39247 req_id = ae32dd7a-fc06-4073-a19a-d2e561f6f68f response_code = 200 uri = /v2p
17402024-08-28T00:57:07.483ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:61375
17412024-08-28T00:57:07.483ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
17422024-08-28T00:57:07.525ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
17432024-08-28T00:57:07.581ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 98414 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:61375 req_id = 63bae451-f72c-407a-8a24-6c9b51d4d51f response_code = 204 uri = /switch-ports
17442024-08-28T00:57:07.581ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:45183
17452024-08-28T00:57:07.581ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
17462024-08-28T00:57:07.585ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3574 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:61375 req_id = 481d42b5-2fc4-49e0-bdad-fb9402c25030 response_code = 200 uri = /network-bootstore-config
17472024-08-28T00:57:07.616ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
17482024-08-28T00:57:07.673ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 91278 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:45183 req_id = 09a2d75f-fc2f-447c-8909-e3fbe004b76d response_code = 204 uri = /switch-ports
17492024-08-28T00:57:07.674ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 764 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:45183 req_id = 2e394321-f381-47ca-a485-ab55dda33290 response_code = 200 uri = /network-bootstore-config
17502024-08-28T00:57:24.480ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:57446
17512024-08-28T00:57:24.481ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 658 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:57446 req_id = 648b7bfe-67a6-4214-b0a2-e2c52c653d25 response_code = 200 uri = /baseboard
17522024-08-28T00:57:36.306ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:64160
17532024-08-28T00:57:36.306ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 93 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:64160 req_id = 22ebbab4-dc40-405f-8e8e-a0fc093bfd8e response_code = 200 uri = /vpc-routes
17542024-08-28T00:57:36.425ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 149 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:64160 req_id = 347fff9b-bd5f-426e-adf2-abff3d015bab response_code = 204 uri = /vpc-routes
17552024-08-28T00:57:36.430ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:56462
17562024-08-28T00:57:36.430ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 125 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:56462 req_id = 928ec48c-1ce3-45ce-bbb6-94acae7e08a5 response_code = 200 uri = /v2p
17572024-08-28T00:57:36.706ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:38544
17582024-08-28T00:57:36.707ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
17592024-08-28T00:57:36.746ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
17602024-08-28T00:57:36.797ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 90471 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:38544 req_id = b6856731-8067-41d5-910d-f94a80d7dfaf response_code = 204 uri = /switch-ports
17612024-08-28T00:57:36.797ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 119 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:38544 req_id = f0446244-6716-40cb-8f33-a630453eb38f response_code = 200 uri = /network-bootstore-config
17622024-08-28T00:57:37.069ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:40920
17632024-08-28T00:57:37.070ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1564 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:40920 req_id = 70e2644a-0ca8-48b0-a80e-6986ff910da9 response_code = 200 uri = /vpc-routes
17642024-08-28T00:57:37.080ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:36902
17652024-08-28T00:57:37.081ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 30 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:36902 req_id = 3f7ca072-c61f-40e9-b93d-3bf45c0c5838 response_code = 200 uri = /vpc-routes
17662024-08-28T00:57:37.144ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:46991
17672024-08-28T00:57:37.144ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 135 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:46991 req_id = 9f26be8a-981f-4318-8ca9-6e1edfb7c5a6 response_code = 200 uri = /v2p
17682024-08-28T00:57:37.183ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 125 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:40920 req_id = ddad6ab7-0621-4ed7-9790-b11b816fed90 response_code = 204 uri = /vpc-routes
17692024-08-28T00:57:37.192ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:64431
17702024-08-28T00:57:37.192ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 94 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:64431 req_id = ecad6497-3317-4cf2-ac6e-cefad6d1874d response_code = 200 uri = /v2p
17712024-08-28T00:57:37.194ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 56 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:36902 req_id = 41dea23b-778e-4e81-a45b-fe73fc577e9c response_code = 204 uri = /vpc-routes
17722024-08-28T00:57:37.302ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:50540
17732024-08-28T00:57:37.303ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
17742024-08-28T00:57:37.343ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
17752024-08-28T00:57:37.402ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 99786 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:50540 req_id = 05ee7a04-8210-41a7-bd27-43f22e674d89 response_code = 204 uri = /switch-ports
17762024-08-28T00:57:37.402ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:54074
17772024-08-28T00:57:37.403ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
17782024-08-28T00:57:37.406ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3674 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:50540 req_id = af9a5907-c36d-4592-bb66-5f966d285f92 response_code = 200 uri = /network-bootstore-config
17792024-08-28T00:57:37.439ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
17802024-08-28T00:57:37.494ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 91248 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:54074 req_id = c0f5c459-5026-4f66-a94d-2f7e387bb2ad response_code = 204 uri = /switch-ports
17812024-08-28T00:57:37.495ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 901 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:54074 req_id = 3371442d-d11c-4c60-ab4d-8cd591be8a27 response_code = 200 uri = /network-bootstore-config
17822024-08-28T00:57:54.512ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:48883
17832024-08-28T00:57:54.512ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 41 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:48883 req_id = 8c774897-7379-4d6d-adfe-c605a3a2ed51 response_code = 200 uri = /baseboard
17842024-08-28T00:58:06.459ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:53143
17852024-08-28T00:58:06.459ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 148 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:53143 req_id = 57d70a1b-ebb9-47b4-9676-959d4d6f3c61 response_code = 200 uri = /v2p
17862024-08-28T00:58:06.509ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:38204
17872024-08-28T00:58:06.509ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 53 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:38204 req_id = aeacf2c4-70ac-44a2-83f2-2bce2283b75f response_code = 200 uri = /vpc-routes
17882024-08-28T00:58:06.617ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:64025
17892024-08-28T00:58:06.617ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
17902024-08-28T00:58:06.658ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
17912024-08-28T00:58:06.722ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 104700 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:64025 req_id = 772268f6-42da-4d68-832d-a460a81e0bcd response_code = 204 uri = /switch-ports
17922024-08-28T00:58:06.722ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 159 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:38204 req_id = 32cfc785-22f9-4ee1-baac-41c203457054 response_code = 204 uri = /vpc-routes
17932024-08-28T00:58:06.722ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 211 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:64025 req_id = ee532aba-5498-4149-84fe-a6790e3b09cb response_code = 200 uri = /network-bootstore-config
17942024-08-28T00:58:07.158ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:43221
17952024-08-28T00:58:07.158ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 40 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:43221 req_id = f8eba0e8-6ec2-4789-b130-7cc83bd19fa1 response_code = 200 uri = /vpc-routes
17962024-08-28T00:58:07.170ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 150 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:43221 req_id = c8be0154-3d6f-40de-bb4f-67097eb4d351 response_code = 204 uri = /vpc-routes
17972024-08-28T00:58:07.310ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:64189
17982024-08-28T00:58:07.310ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 63 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:64189 req_id = d4005335-a759-4362-af58-cb9c7c2bdd13 response_code = 200 uri = /vpc-routes
17992024-08-28T00:58:07.386ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:43464
18002024-08-28T00:58:07.386ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 153 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:43464 req_id = 9e312012-afa6-46a9-a062-2c326390308c response_code = 200 uri = /v2p
18012024-08-28T00:58:07.419ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 107 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:64189 req_id = bf83ff62-bb3a-4c78-9b0c-a03db7cb8246 response_code = 204 uri = /vpc-routes
18022024-08-28T00:58:07.439ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:49491
18032024-08-28T00:58:07.439ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 123 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:49491 req_id = 2d2d12fc-081b-4fe2-a34d-a6c5ae91fc90 response_code = 200 uri = /v2p
18042024-08-28T00:58:07.486ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:35437
18052024-08-28T00:58:07.486ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
18062024-08-28T00:58:07.524ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
18072024-08-28T00:58:07.578ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 92074 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:35437 req_id = 631140c8-2716-4b0e-b697-3f73e1376605 response_code = 204 uri = /switch-ports
18082024-08-28T00:58:07.578ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:57805
18092024-08-28T00:58:07.578ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
18102024-08-28T00:58:07.582ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 4404 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:35437 req_id = f1ba791c-a63e-4dfe-8748-7ea9ab99d864 response_code = 200 uri = /network-bootstore-config
18112024-08-28T00:58:07.614ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
18122024-08-28T00:58:07.667ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 89310 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:57805 req_id = b1951079-77df-47b7-8356-71f38927243a response_code = 204 uri = /switch-ports
18132024-08-28T00:58:07.668ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 172 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:57805 req_id = 35f82267-f380-4068-82e1-39efe57afc98 response_code = 200 uri = /network-bootstore-config
18142024-08-28T00:58:18.750ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 220.577597582s error = "Error Response: status: 503 Service Unavailable; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"eae969aa-5f5d-42b8-833a-0ca067cbdc69\\", \\"content-length\\": \\"133\\", \\"date\\": \\"Wed, 28 Aug 2024 00:58:18 GMT\\"}; value: Error { error_code: Some(\\"ServiceNotAvailable\\"), message: \\"Service Unavailable\\", request_id: \\"eae969aa-5f5d-42b8-833a-0ca067cbdc69\\" }" file = oximeter/producer/src/lib.rs:424 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
18152024-08-28T00:58:24.544ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:34432
18162024-08-28T00:58:24.544ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 94 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:34432 req_id = 0a3b8fa7-d6bf-4490-91b4-d00c7365b532 response_code = 200 uri = /baseboard
18172024-08-28T00:58:36.366ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:54508
18182024-08-28T00:58:36.367ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 678 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:54508 req_id = 474b7d21-2a6b-4be9-b4fc-5f9bf024510d response_code = 200 uri = /vpc-routes
18192024-08-28T00:58:36.479ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 123 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:54508 req_id = a41f390b-8e77-43ef-b87e-8b02387ec71f response_code = 204 uri = /vpc-routes
18202024-08-28T00:58:36.497ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:40602
18212024-08-28T00:58:36.497ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 95 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:40602 req_id = 150af2c2-0025-4d15-a06f-2b1338d17243 response_code = 200 uri = /v2p
18222024-08-28T00:58:36.622ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:63191
18232024-08-28T00:58:36.622ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
18242024-08-28T00:58:36.660ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
18252024-08-28T00:58:36.718ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 95997 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:63191 req_id = 474eb222-aa94-4202-9789-2b7976582948 response_code = 204 uri = /switch-ports
18262024-08-28T00:58:36.718ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 122 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:63191 req_id = 5b4f1241-6959-475a-b794-660cb61181b9 response_code = 200 uri = /network-bootstore-config
18272024-08-28T00:58:37.022ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:57533
18282024-08-28T00:58:37.022ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 191 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:57533 req_id = bd87badb-6e33-40ac-a013-2f1f0da32728 response_code = 200 uri = /v2p
18292024-08-28T00:58:37.055ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:35363
18302024-08-28T00:58:37.055ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 48 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:35363 req_id = e6ebc826-2ec7-456e-adf2-4bbfc1d5cc2b response_code = 200 uri = /vpc-routes
18312024-08-28T00:58:37.062ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:63061
18322024-08-28T00:58:37.062ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 29 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:63061 req_id = 8df0fd62-d696-426f-91a4-3c0657a27523 response_code = 200 uri = /vpc-routes
18332024-08-28T00:58:37.068ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 114 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:35363 req_id = 1a549b6b-49f6-4e34-bfac-905875060006 response_code = 204 uri = /vpc-routes
18342024-08-28T00:58:37.077ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 897 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:63061 req_id = 22b29aac-ab2d-4648-b559-78baae1b1ab3 response_code = 204 uri = /vpc-routes
18352024-08-28T00:58:37.162ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:47277
18362024-08-28T00:58:37.162ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 124 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:47277 req_id = 1a6e4808-ecbd-4c97-b80a-73eda89cbd2c response_code = 200 uri = /v2p
18372024-08-28T00:58:37.237ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:34769
18382024-08-28T00:58:37.237ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
18392024-08-28T00:58:37.273ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
18402024-08-28T00:58:37.328ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 90914 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:34769 req_id = 9d9142f7-5bae-4afc-a88f-a81b2170f1e2 response_code = 204 uri = /switch-ports
18412024-08-28T00:58:37.328ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:61024
18422024-08-28T00:58:37.329ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
18432024-08-28T00:58:37.332ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 4139 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:34769 req_id = 058f4e18-c72b-4284-af59-03f0a06fd6d2 response_code = 200 uri = /network-bootstore-config
18442024-08-28T00:58:37.364ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
18452024-08-28T00:58:37.421ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 92882 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:61024 req_id = 3fc7d4c2-a4d3-40e1-a204-91633bc394dc response_code = 204 uri = /switch-ports
18462024-08-28T00:58:37.421ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 181 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:61024 req_id = 3881834b-8a71-4aea-9abe-49de2b92cd8f response_code = 200 uri = /network-bootstore-config
18472024-08-28T00:58:54.578ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:41710
18482024-08-28T00:58:54.578ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 44 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:41710 req_id = 342ad510-55cb-45e2-8545-523b071a010e response_code = 200 uri = /baseboard
18492024-08-28T00:59:06.528ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:49779
18502024-08-28T00:59:06.528ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 55 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:49779 req_id = 1d8fe407-f6e8-4ad5-a553-1f0f7fe86eb5 response_code = 200 uri = /vpc-routes
18512024-08-28T00:59:06.615ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:53723
18522024-08-28T00:59:06.615ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 139 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:53723 req_id = 5c3dc53b-0722-487b-906f-5e05235ac43f response_code = 200 uri = /v2p
18532024-08-28T00:59:06.647ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 698 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:49779 req_id = 94f82c36-844f-47bd-b89b-57d3c6be0a8f response_code = 204 uri = /vpc-routes
18542024-08-28T00:59:06.701ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:57514
18552024-08-28T00:59:06.701ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
18562024-08-28T00:59:06.741ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
18572024-08-28T00:59:06.800ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 99163 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:57514 req_id = c1d7413e-26d3-4919-ad0d-f3246ec41c35 response_code = 204 uri = /switch-ports
18582024-08-28T00:59:06.800ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 107 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:57514 req_id = 77a749c7-94a1-4d84-85f8-ee910d4f9888 response_code = 200 uri = /network-bootstore-config
18592024-08-28T00:59:07.064ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:47535
18602024-08-28T00:59:07.065ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1047 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:47535 req_id = 33065ea9-6925-4c7a-81c9-16692dd404e6 response_code = 200 uri = /v2p
18612024-08-28T00:59:07.106ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:43216
18622024-08-28T00:59:07.106ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 25 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:43216 req_id = 7fcc42e5-1fc4-44cf-a393-4af815731259 response_code = 200 uri = /vpc-routes
18632024-08-28T00:59:07.128ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 111 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:43216 req_id = cbdc32e7-a4ae-4b6b-9458-ced811d6ec86 response_code = 204 uri = /vpc-routes
18642024-08-28T00:59:07.208ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:35050
18652024-08-28T00:59:07.208ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 35 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:35050 req_id = 9c2fb55a-7c5b-4092-9312-24832c86ef0f response_code = 200 uri = /vpc-routes
18662024-08-28T00:59:07.225ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 121 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:35050 req_id = 926f24d4-5276-4f68-81fb-fe444166c0b0 response_code = 204 uri = /vpc-routes
18672024-08-28T00:59:07.319ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:44547
18682024-08-28T00:59:07.319ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 127 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:44547 req_id = f887483d-110a-42c4-98d7-5e931a1fbffa response_code = 200 uri = /v2p
18692024-08-28T00:59:07.407ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:47328
18702024-08-28T00:59:07.407ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
18712024-08-28T00:59:07.445ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
18722024-08-28T00:59:07.497ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 89913 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:47328 req_id = 29d8c96f-f0a2-4e55-a59b-bfa200aaa7cd response_code = 204 uri = /switch-ports
18732024-08-28T00:59:07.497ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 116 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:47328 req_id = 32f4b10b-831c-47e7-9bd3-2a94ce04185e response_code = 200 uri = /network-bootstore-config
18742024-08-28T00:59:07.552ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:61203
18752024-08-28T00:59:07.552ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
18762024-08-28T00:59:07.590ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
18772024-08-28T00:59:07.646ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 93409 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:61203 req_id = 763940dc-dbe6-48c0-a78e-0be896f41711 response_code = 204 uri = /switch-ports
18782024-08-28T00:59:07.646ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 99 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:61203 req_id = 19f10e6d-d4b3-4c1a-a0f6-9be3273fdd56 response_code = 200 uri = /network-bootstore-config
18792024-08-28T00:59:24.611ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:43232
18802024-08-28T00:59:24.611ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 39 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:43232 req_id = ea74048b-7660-4921-9471-23795b222c90 response_code = 200 uri = /baseboard
18812024-08-28T00:59:36.529ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:33127
18822024-08-28T00:59:36.529ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 62 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:33127 req_id = 2f03b73a-6268-47d6-9945-4b25b4971e79 response_code = 200 uri = /vpc-routes
18832024-08-28T00:59:36.546ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 792 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:33127 req_id = d7526c0e-2bb9-4172-b200-9cb03c2a8c67 response_code = 204 uri = /vpc-routes
18842024-08-28T00:59:36.608ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:54922
18852024-08-28T00:59:36.608ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 127 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:54922 req_id = fdca7d8f-f453-4e6a-850e-fe88c2ff341a response_code = 200 uri = /v2p
18862024-08-28T00:59:36.631ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:34818
18872024-08-28T00:59:36.632ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
18882024-08-28T00:59:36.674ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
18892024-08-28T00:59:36.735ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 103273 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:34818 req_id = 5c3c3b51-730f-4c91-b98e-a3184a1af629 response_code = 204 uri = /switch-ports
18902024-08-28T00:59:36.736ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 848 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:34818 req_id = 16aea49a-042c-4515-9dfb-dbf5ee0cfa9a response_code = 200 uri = /network-bootstore-config
18912024-08-28T00:59:37.038ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:45296
18922024-08-28T00:59:37.038ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 739 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:45296 req_id = 7f1a3bfc-5874-4ec7-a399-bce70e3aa1a4 response_code = 200 uri = /vpc-routes
18932024-08-28T00:59:37.048ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 170 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:45296 req_id = ad17bbea-5365-49e4-9d58-d6dd93fde8a2 response_code = 204 uri = /vpc-routes
18942024-08-28T00:59:37.051ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:40246
18952024-08-28T00:59:37.051ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 43 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:40246 req_id = 82da3261-64aa-483d-b5b0-325d43691990 response_code = 200 uri = /vpc-routes
18962024-08-28T00:59:37.062ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 88 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:40246 req_id = 3a3dccc5-499f-4a3d-ada3-654c1ad42f11 response_code = 204 uri = /vpc-routes
18972024-08-28T00:59:37.121ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:38840
18982024-08-28T00:59:37.122ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 108 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:38840 req_id = 0a3f7588-8417-4722-b9a0-0aadeda6abc2 response_code = 200 uri = /v2p
18992024-08-28T00:59:37.130ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:50225
19002024-08-28T00:59:37.130ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
19012024-08-28T00:59:37.170ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
19022024-08-28T00:59:37.232ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 101372 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:50225 req_id = 1b0f99d7-8097-440c-8635-6011e1f586ef response_code = 204 uri = /switch-ports
19032024-08-28T00:59:37.232ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:45699
19042024-08-28T00:59:37.232ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 194 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:50225 req_id = 4ee655c1-fa5a-4574-907f-2c98c34ee5c5 response_code = 200 uri = /network-bootstore-config
19052024-08-28T00:59:37.233ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1174 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:45699 req_id = 3b1a7329-a2cd-4c79-aa67-9249ab838bed response_code = 200 uri = /v2p
19062024-08-28T00:59:37.361ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:38714
19072024-08-28T00:59:37.361ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
19082024-08-28T00:59:37.402ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
19092024-08-28T00:59:37.463ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 102245 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:38714 req_id = 63781ca9-a866-4157-8985-81c0b7c5936d response_code = 204 uri = /switch-ports
19102024-08-28T00:59:37.464ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 111 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:38714 req_id = efed471a-95c5-455b-81e1-fd76f3b47636 response_code = 200 uri = /network-bootstore-config
19112024-08-28T00:59:54.643ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:61678
19122024-08-28T00:59:54.643ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 94 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:61678 req_id = 6974caee-2a76-4456-9598-62cdb4306af6 response_code = 200 uri = /baseboard
19132024-08-28T01:00:06.432ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:46784
19142024-08-28T01:00:06.432ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 104 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:46784 req_id = fbff9fe5-9b05-405e-85eb-dd5ff8aa490c response_code = 200 uri = /vpc-routes
19152024-08-28T01:00:06.463ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:51876
19162024-08-28T01:00:06.463ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 155 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:51876 req_id = 190c647a-36f8-41d5-a6bc-b2cd08514385 response_code = 200 uri = /v2p
19172024-08-28T01:00:06.473ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 699 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:46784 req_id = 1176abe2-9003-4778-a5ef-75e097325633 response_code = 204 uri = /vpc-routes
19182024-08-28T01:00:06.690ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:62754
19192024-08-28T01:00:06.691ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
19202024-08-28T01:00:06.735ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
19212024-08-28T01:00:06.792ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 101407 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:62754 req_id = 4e507df8-4a8a-40d6-aba2-71883b973c34 response_code = 204 uri = /switch-ports
19222024-08-28T01:00:06.792ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 117 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:62754 req_id = 932be577-d7d3-4c0a-bc0e-cb350bc38b46 response_code = 200 uri = /network-bootstore-config
19232024-08-28T01:00:07.072ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:36825
19242024-08-28T01:00:07.073ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1192 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:36825 req_id = aea0431b-2745-4efa-a725-eabd1d672985 response_code = 200 uri = /v2p
19252024-08-28T01:00:07.114ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:53689
19262024-08-28T01:00:07.114ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 51 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:53689 req_id = 2fa08d8a-382b-4309-8e68-d626d387b221 response_code = 200 uri = /vpc-routes
19272024-08-28T01:00:07.122ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 114 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:53689 req_id = 3d60d23e-4156-4623-827d-ee7df50fd7c9 response_code = 204 uri = /vpc-routes
19282024-08-28T01:00:07.231ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:53537
19292024-08-28T01:00:07.232ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1039 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:53537 req_id = 3842f172-9bf3-469d-9bbb-ff64aeb72ddb response_code = 200 uri = /vpc-routes
19302024-08-28T01:00:07.241ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 87 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:53537 req_id = aa9cc432-e90f-404d-ab57-ba00871eb25f response_code = 204 uri = /vpc-routes
19312024-08-28T01:00:07.257ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:50583
19322024-08-28T01:00:07.257ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 98 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:50583 req_id = 7552ad08-d403-4bbe-a323-5574186ae236 response_code = 200 uri = /v2p
19332024-08-28T01:00:07.316ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:54473
19342024-08-28T01:00:07.317ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
19352024-08-28T01:00:07.361ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
19362024-08-28T01:00:07.419ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 102518 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:54473 req_id = fd7a1508-e93d-481f-98fa-eef366b59c79 response_code = 204 uri = /switch-ports
19372024-08-28T01:00:07.419ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:64382
19382024-08-28T01:00:07.419ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
19392024-08-28T01:00:07.454ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
19402024-08-28T01:00:07.509ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 89905 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:64382 req_id = 6996b96a-ff64-4f6f-88c2-371d015b3dac response_code = 204 uri = /switch-ports
19412024-08-28T01:00:07.509ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 97 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:54473 req_id = f15fdd06-f68d-4a57-94c9-7e882ed11ba5 response_code = 200 uri = /network-bootstore-config
19422024-08-28T01:00:07.509ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 157 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:64382 req_id = 3336d5e5-49d0-429f-9c29-c4ce8dfb2729 response_code = 200 uri = /network-bootstore-config
19432024-08-28T01:00:24.677ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:62257
19442024-08-28T01:00:24.677ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 50 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:62257 req_id = 818683b0-9d66-46c1-a9ca-e92e23c710bf response_code = 200 uri = /baseboard
19452024-08-28T01:00:36.374ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:51122
19462024-08-28T01:00:36.374ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 93 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:51122 req_id = 65894715-8d5a-4b7d-be93-7412629d8cf6 response_code = 200 uri = /vpc-routes
19472024-08-28T01:00:36.383ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 128 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:51122 req_id = 7ab05c40-dc3e-404e-bfcc-fd46bf066e6d response_code = 204 uri = /vpc-routes
19482024-08-28T01:00:36.458ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:55087
19492024-08-28T01:00:36.459ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 134 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:55087 req_id = f2a6d9dd-8801-4c4d-abea-1e0c1de07f69 response_code = 200 uri = /v2p
19502024-08-28T01:00:36.505ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:36256
19512024-08-28T01:00:36.505ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
19522024-08-28T01:00:36.544ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
19532024-08-28T01:00:36.603ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 97341 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:36256 req_id = fc75241c-438e-49b0-8ade-9c4de420200c response_code = 204 uri = /switch-ports
19542024-08-28T01:00:36.603ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 201 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:36256 req_id = 9eb7643d-9d79-4789-8a34-e84469fdc530 response_code = 200 uri = /network-bootstore-config
19552024-08-28T01:00:36.994ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:38734
19562024-08-28T01:00:36.994ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 61 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:38734 req_id = e6ad0d0c-c4eb-48dc-8403-f381b89ad539 response_code = 200 uri = /vpc-routes
19572024-08-28T01:00:37.004ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 105 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:38734 req_id = 66e9a304-b522-43f1-84ab-2222e288ec00 response_code = 204 uri = /vpc-routes
19582024-08-28T01:00:37.051ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:52602
19592024-08-28T01:00:37.051ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 38 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:52602 req_id = 200537ef-aeaf-423a-b896-302c2fac5e96 response_code = 200 uri = /vpc-routes
19602024-08-28T01:00:37.061ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 93 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:52602 req_id = 78431452-6493-4364-8478-0b20cfef8780 response_code = 204 uri = /vpc-routes
19612024-08-28T01:00:37.122ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:60524
19622024-08-28T01:00:37.122ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
19632024-08-28T01:00:37.164ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
19642024-08-28T01:00:37.226ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 103438 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:60524 req_id = 6e619bfa-b8cc-45d0-ae68-50e6c413d9fd response_code = 204 uri = /switch-ports
19652024-08-28T01:00:37.226ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:58035
19662024-08-28T01:00:37.226ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:46174
19672024-08-28T01:00:37.226ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 206 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:58035 req_id = be90c051-fb0e-4e60-bcb1-ea8659979e26 response_code = 200 uri = /v2p
19682024-08-28T01:00:37.226ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
19692024-08-28T01:00:37.226ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 274 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:60524 req_id = 80c0cc3e-c01c-4b73-ac76-3aab7f3eb3a9 response_code = 200 uri = /network-bootstore-config
19702024-08-28T01:00:37.268ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
19712024-08-28T01:00:37.277ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:49743
19722024-08-28T01:00:37.277ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 178 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:49743 req_id = 8abca13a-964e-456d-a4e8-7a069568384e response_code = 200 uri = /v2p
19732024-08-28T01:00:37.326ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 99404 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:46174 req_id = b0a02b7b-19da-4831-ab2e-16b9594c7aec response_code = 204 uri = /switch-ports
19742024-08-28T01:00:37.326ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 152 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:46174 req_id = f74328d1-21d7-433a-b5a4-99e9973a9a53 response_code = 200 uri = /network-bootstore-config
19752024-08-28T01:00:54.710ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:44312
19762024-08-28T01:00:54.711ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 872 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:44312 req_id = fab5112e-5323-4efb-bbb6-1240dd2f13dc response_code = 200 uri = /baseboard
19772024-08-28T01:01:06.452ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:36479
19782024-08-28T01:01:06.452ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 122 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:36479 req_id = 2598728d-492a-4b47-bc82-2ca564d2b89d response_code = 200 uri = /vpc-routes
19792024-08-28T01:01:06.467ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 164 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:36479 req_id = 46482c77-3511-486f-bbab-d8942e406c88 response_code = 204 uri = /vpc-routes
19802024-08-28T01:01:06.617ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:45419
19812024-08-28T01:01:06.617ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 159 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:45419 req_id = 7b6056b0-b2dc-4ac9-8d22-6d9bcfb03b80 response_code = 200 uri = /v2p
19822024-08-28T01:01:06.639ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:56789
19832024-08-28T01:01:06.640ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
19842024-08-28T01:01:06.683ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
19852024-08-28T01:01:06.746ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 107496 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:56789 req_id = 14d916bb-c481-4b9c-aa57-d664b8829010 response_code = 204 uri = /switch-ports
19862024-08-28T01:01:06.747ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 110 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:56789 req_id = ec9d8a21-9462-443a-a4d7-e0ed1dc3e895 response_code = 200 uri = /network-bootstore-config
19872024-08-28T01:01:06.752ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:39852
19882024-08-28T01:01:06.752ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 1eb980bf-46dc-45a7-aa4f-098bf58d6904, kind: Service { id: 2cae9f85-8043-498f-af8e-a4bc9724ac55 }, name: Name("external-dns-2cae9f85-8043-498f-af8e-a4bc9724ac55"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 212, 38])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: eafd744b-ffb5-445c-8366-bd33299374a6, kind: Service { id: fa9e7556-b41e-4114-bf3f-e918820a9bcf }, name: Name("external-dns-fa9e7556-b41e-4114-bf3f-e918820a9bcf"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 202, 132])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 16870a00-56de-4335-bbe4-e10f8544f208, kind: Service { id: f840e071-026c-488d-b1c9-2c4d1ce3e74d }, name: Name("nexus-f840e071-026c-488d-b1c9-2c4d1ce3e74d"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 209, 34])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 707af25c-dc9c-42b2-9351-0dc3824f26c9, kind: Service { id: a5227819-155d-4a08-8e71-bd8a327ad739 }, name: Name("nexus-a5227819-155d-4a08-8e71-bd8a327ad739"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 220, 190])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a6310629-11e4-4b6d-b328-1d8767a4ab69, kind: Service { id: 4ee19452-709a-4abd-a238-7af80f61d166 }, name: Name("nexus-4ee19452-709a-4abd-a238-7af80f61d166"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 191, 68])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
19892024-08-28T01:01:06.752ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19902024-08-28T01:01:06.752ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
19912024-08-28T01:01:06.752ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19922024-08-28T01:01:06.752ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
19932024-08-28T01:01:06.752ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19942024-08-28T01:01:06.752ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
19952024-08-28T01:01:06.753ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 365 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:39852 req_id = 8b4124c2-f123-46b4-8999-703e48e49e46 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
19962024-08-28T01:01:07.264ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:61506
19972024-08-28T01:01:07.265ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 915 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:61506 req_id = c9986647-2816-49a6-9367-b55e8811240b response_code = 200 uri = /v2p
19982024-08-28T01:01:07.296ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:45262
19992024-08-28T01:01:07.296ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 31 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:45262 req_id = 44cdb717-8085-4395-9510-255f8f9001f8 response_code = 200 uri = /vpc-routes
20002024-08-28T01:01:07.303ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:33777
20012024-08-28T01:01:07.303ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 19 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:33777 req_id = b8e41b64-7662-48ca-895a-a0a3c921a89e response_code = 200 uri = /vpc-routes
20022024-08-28T01:01:07.304ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 84 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:45262 req_id = 7905d175-b3d0-4c17-9b30-da8838395d07 response_code = 204 uri = /vpc-routes
20032024-08-28T01:01:07.316ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 134 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:33777 req_id = 369cf011-cb8a-44ce-b14e-844c92e1aa02 response_code = 204 uri = /vpc-routes
20042024-08-28T01:01:07.360ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:44038
20052024-08-28T01:01:07.360ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
20062024-08-28T01:01:07.405ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
20072024-08-28T01:01:07.472ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 111152 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:44038 req_id = 8ea2506a-12c1-4931-987c-372b97aeed11 response_code = 204 uri = /switch-ports
20082024-08-28T01:01:07.472ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:35172
20092024-08-28T01:01:07.472ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
20102024-08-28T01:01:07.512ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
20112024-08-28T01:01:07.571ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 99343 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:35172 req_id = bc23e38c-94d9-404f-8437-144a51199580 response_code = 204 uri = /switch-ports
20122024-08-28T01:01:07.571ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 158 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:44038 req_id = 3cacfd85-8791-4b04-b8fc-d4466a808288 response_code = 200 uri = /network-bootstore-config
20132024-08-28T01:01:07.571ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:48146
20142024-08-28T01:01:07.571ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:46023
20152024-08-28T01:01:07.572ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 299 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:35172 req_id = efb3017b-97d0-44e5-986e-9ae673170cf1 response_code = 200 uri = /network-bootstore-config
20162024-08-28T01:01:07.572ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 268 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:46023 req_id = d9b81c4d-7776-4275-8db9-d902d2460859 response_code = 200 uri = /v2p
20172024-08-28T01:01:07.572ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 1eb980bf-46dc-45a7-aa4f-098bf58d6904, kind: Service { id: 2cae9f85-8043-498f-af8e-a4bc9724ac55 }, name: Name("external-dns-2cae9f85-8043-498f-af8e-a4bc9724ac55"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 212, 38])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: eafd744b-ffb5-445c-8366-bd33299374a6, kind: Service { id: fa9e7556-b41e-4114-bf3f-e918820a9bcf }, name: Name("external-dns-fa9e7556-b41e-4114-bf3f-e918820a9bcf"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 202, 132])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 16870a00-56de-4335-bbe4-e10f8544f208, kind: Service { id: f840e071-026c-488d-b1c9-2c4d1ce3e74d }, name: Name("nexus-f840e071-026c-488d-b1c9-2c4d1ce3e74d"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 209, 34])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 707af25c-dc9c-42b2-9351-0dc3824f26c9, kind: Service { id: a5227819-155d-4a08-8e71-bd8a327ad739 }, name: Name("nexus-a5227819-155d-4a08-8e71-bd8a327ad739"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 220, 190])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a6310629-11e4-4b6d-b328-1d8767a4ab69, kind: Service { id: 4ee19452-709a-4abd-a238-7af80f61d166 }, name: Name("nexus-4ee19452-709a-4abd-a238-7af80f61d166"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 191, 68])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
20182024-08-28T01:01:07.572ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
20192024-08-28T01:01:07.572ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
20202024-08-28T01:01:07.572ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
20212024-08-28T01:01:07.572ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
20222024-08-28T01:01:07.572ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
20232024-08-28T01:01:07.572ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
20242024-08-28T01:01:07.572ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 414 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:48146 req_id = 0315f965-89a5-4a9c-9a46-a45d7efed0e1 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
20252024-08-28T01:01:07.610ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:53939
20262024-08-28T01:01:07.610ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 1eb980bf-46dc-45a7-aa4f-098bf58d6904, kind: Service { id: 2cae9f85-8043-498f-af8e-a4bc9724ac55 }, name: Name("external-dns-2cae9f85-8043-498f-af8e-a4bc9724ac55"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 212, 38])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: eafd744b-ffb5-445c-8366-bd33299374a6, kind: Service { id: fa9e7556-b41e-4114-bf3f-e918820a9bcf }, name: Name("external-dns-fa9e7556-b41e-4114-bf3f-e918820a9bcf"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 202, 132])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 16870a00-56de-4335-bbe4-e10f8544f208, kind: Service { id: f840e071-026c-488d-b1c9-2c4d1ce3e74d }, name: Name("nexus-f840e071-026c-488d-b1c9-2c4d1ce3e74d"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 209, 34])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 707af25c-dc9c-42b2-9351-0dc3824f26c9, kind: Service { id: a5227819-155d-4a08-8e71-bd8a327ad739 }, name: Name("nexus-a5227819-155d-4a08-8e71-bd8a327ad739"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 220, 190])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a6310629-11e4-4b6d-b328-1d8767a4ab69, kind: Service { id: 4ee19452-709a-4abd-a238-7af80f61d166 }, name: Name("nexus-4ee19452-709a-4abd-a238-7af80f61d166"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 191, 68])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
20272024-08-28T01:01:07.610ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
20282024-08-28T01:01:07.611ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
20292024-08-28T01:01:07.611ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
20302024-08-28T01:01:07.611ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
20312024-08-28T01:01:07.611ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
20322024-08-28T01:01:07.611ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
20332024-08-28T01:01:07.611ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 913 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:53939 req_id = 7ca54227-9261-46ea-845a-e343e34945f3 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
20342024-08-28T01:01:24.744ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:61481
20352024-08-28T01:01:24.744ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 575 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:61481 req_id = cf804610-a3ea-4a03-ab1d-f8433f11644e response_code = 200 uri = /baseboard
20362024-08-28T01:01:30.428ZINFOSledAgent (auto-cleanup-task): running automatic periodic zone bundle cleanup file = sled-agent/src/zone_bundle.rs:197
20372024-08-28T01:01:30.428ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
20382024-08-28T01:01:30.474ZINFOSledAgent (auto-cleanup-task): top of bundle cleanup loop file = sled-agent/src/zone_bundle.rs:186 next_cleanup = Instant { tv_sec: 1953, tv_nsec: 988196556 } time_to_next_cleanup = 599.99999749s
20392024-08-28T01:01:36.451ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:64813
20402024-08-28T01:01:36.451ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 81 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:64813 req_id = 8da8ee08-15ea-49b5-bfef-c290fafe3dae response_code = 200 uri = /vpc-routes
20412024-08-28T01:01:36.463ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 826 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:64813 req_id = 7435e3f0-e355-4ec4-8e20-3f967f2fd068 response_code = 204 uri = /vpc-routes
20422024-08-28T01:01:36.562ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:53771
20432024-08-28T01:01:36.562ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 121 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:53771 req_id = 04e49e3a-7591-443c-bf52-adc2bc8f6518 response_code = 200 uri = /v2p
20442024-08-28T01:01:36.577ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:47204
20452024-08-28T01:01:36.577ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
20462024-08-28T01:01:36.621ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
20472024-08-28T01:01:36.679ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 101970 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:47204 req_id = aa9d2e0a-fde1-406d-b66c-f7551b2e59b5 response_code = 204 uri = /switch-ports
20482024-08-28T01:01:36.679ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 207 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:47204 req_id = 0e11065d-f0f5-4a95-a08c-a14125bc7d1d response_code = 200 uri = /network-bootstore-config
20492024-08-28T01:01:37.048ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:52582
20502024-08-28T01:01:37.048ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 92 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:52582 req_id = f6ab954c-bbff-4272-a5f6-e6a1ebeb6ba9 response_code = 200 uri = /vpc-routes
20512024-08-28T01:01:37.058ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 126 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:52582 req_id = 9bf66aed-edda-4044-883f-832a4b343ad4 response_code = 204 uri = /vpc-routes
20522024-08-28T01:01:37.129ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:64634
20532024-08-28T01:01:37.129ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 141 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:64634 req_id = f9b8a428-9978-454f-a84c-5fee536b79bc response_code = 200 uri = /v2p
20542024-08-28T01:01:37.192ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:40437
20552024-08-28T01:01:37.192ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
20562024-08-28T01:01:37.234ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
20572024-08-28T01:01:37.298ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 105947 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:40437 req_id = c69f40fc-35d1-4eb8-9b3c-f4a26c6b7886 response_code = 204 uri = /switch-ports
20582024-08-28T01:01:37.298ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:43416
20592024-08-28T01:01:37.298ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:64048
20602024-08-28T01:01:37.298ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:57526
20612024-08-28T01:01:37.298ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 394 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:40437 req_id = f5ca25c6-9365-46d5-9a27-4ed7c3ff091f response_code = 200 uri = /network-bootstore-config
20622024-08-28T01:01:37.298ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 167 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:64048 req_id = 4631f90c-5a1b-44fe-8cc1-c822110e030f response_code = 200 uri = /vpc-routes
20632024-08-28T01:01:37.298ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 287 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:43416 req_id = a387a6a1-e4ea-458c-ba39-74f16eaa64ec response_code = 200 uri = /v2p
20642024-08-28T01:01:37.299ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
20652024-08-28T01:01:37.307ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 281 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:64048 req_id = f91e5846-0402-4342-8aab-204bf0d95425 response_code = 204 uri = /vpc-routes
20662024-08-28T01:01:37.340ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
20672024-08-28T01:01:37.393ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 94617 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:57526 req_id = 21cf9245-436a-451f-a9cc-8ed301830a6c response_code = 204 uri = /switch-ports
20682024-08-28T01:01:37.393ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 178 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:57526 req_id = 0a4686cc-fd28-41e8-ad03-5ec9b2addd89 response_code = 200 uri = /network-bootstore-config
20692024-08-28T01:01:44.053ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
20702024-08-28T01:01:54.778ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:41580
20712024-08-28T01:01:54.778ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 45 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:41580 req_id = c81ea960-0dcc-4ade-80db-48b6b9b798a0 response_code = 200 uri = /baseboard
20722024-08-28T01:01:59.368ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 239.399762795s error = "Error Response: status: 503 Service Unavailable; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"f3f552d0-e013-4c5d-bb2d-a9302baf971a\\", \\"content-length\\": \\"133\\", \\"date\\": \\"Wed, 28 Aug 2024 01:01:59 GMT\\"}; value: Error { error_code: Some(\\"ServiceNotAvailable\\"), message: \\"Service Unavailable\\", request_id: \\"f3f552d0-e013-4c5d-bb2d-a9302baf971a\\" }" file = oximeter/producer/src/lib.rs:424 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
20732024-08-28T01:02:06.434ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:55444
20742024-08-28T01:02:06.435ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 264 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:55444 req_id = 0bfb0306-5bc4-4a7f-996e-40495e3af36e response_code = 200 uri = /vpc-routes
20752024-08-28T01:02:06.446ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 129 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:55444 req_id = 8e09dc96-a8a2-451d-91c3-f254a20cd4d8 response_code = 204 uri = /vpc-routes
20762024-08-28T01:02:06.598ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:57919
20772024-08-28T01:02:06.598ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 153 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:57919 req_id = 251db403-59df-4b43-b3cc-b33eff58316b response_code = 200 uri = /v2p
20782024-08-28T01:02:06.670ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:60627
20792024-08-28T01:02:06.670ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
20802024-08-28T01:02:06.713ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
20812024-08-28T01:02:06.774ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 104222 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:60627 req_id = c5dfe463-a464-473d-a766-7ddddcb551b3 response_code = 204 uri = /switch-ports
20822024-08-28T01:02:06.774ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 197 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:60627 req_id = 6f16d29e-a328-4dda-a247-4dbafdc5f6d7 response_code = 200 uri = /network-bootstore-config
20832024-08-28T01:02:07.274ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51963
20842024-08-28T01:02:07.274ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 156 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:51963 req_id = 6f823f43-c915-417b-b42c-305234606824 response_code = 200 uri = /v2p
20852024-08-28T01:02:07.321ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:39872
20862024-08-28T01:02:07.322ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 145 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:39872 req_id = 00c10398-e13a-441f-8c32-c8981197b788 response_code = 200 uri = /v2p
20872024-08-28T01:02:07.323ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:43836
20882024-08-28T01:02:07.323ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 33 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:43836 req_id = 4e5444ee-7d40-437b-b5a8-6bdefcb08f07 response_code = 200 uri = /vpc-routes
20892024-08-28T01:02:07.334ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 134 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:43836 req_id = 70801eb9-6150-47c3-8e74-c1cd09b8cec2 response_code = 204 uri = /vpc-routes
20902024-08-28T01:02:07.394ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:57195
20912024-08-28T01:02:07.394ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 31 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:57195 req_id = 1404f2b8-fdd7-4dd4-9a4a-f29fa8d1d32f response_code = 200 uri = /vpc-routes
20922024-08-28T01:02:07.397ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:36156
20932024-08-28T01:02:07.397ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
20942024-08-28T01:02:07.442ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
20952024-08-28T01:02:07.504ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 106480 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:36156 req_id = 51de7044-9397-4d2c-b532-965cc04e7402 response_code = 204 uri = /switch-ports
20962024-08-28T01:02:07.504ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:34370
20972024-08-28T01:02:07.504ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 191 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:57195 req_id = 269b65e8-ef41-45e8-a12b-593746bd3f27 response_code = 204 uri = /vpc-routes
20982024-08-28T01:02:07.504ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 273 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:36156 req_id = 3f43d47c-a943-489b-a6bb-e05a9d88d525 response_code = 200 uri = /network-bootstore-config
20992024-08-28T01:02:07.504ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
21002024-08-28T01:02:07.547ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
21012024-08-28T01:02:07.605ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 100999 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:34370 req_id = ac398e62-ea95-4ef3-be1b-8dd67cb48b67 response_code = 204 uri = /switch-ports
21022024-08-28T01:02:07.606ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 611 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:34370 req_id = 63cb6477-5027-42ea-8f4b-e42dd61a33e2 response_code = 200 uri = /network-bootstore-config
21032024-08-28T01:02:24.827ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:56090
21042024-08-28T01:02:24.827ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 63 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:56090 req_id = 68435f5e-349b-44e6-91df-79c5244eaec8 response_code = 200 uri = /baseboard
21052024-08-28T01:02:33.419ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:40427
21062024-08-28T01:02:33.419ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 134 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:40427 req_id = ff775316-d1a4-40c9-a41c-2da0a40fbde5 response_code = 200 uri = /v2p
21072024-08-28T01:02:33.420ZINFOSledAgent (PortManager): Mapping virtual NIC to physical host file = illumos-utils/src/opte/port_manager.rs:778 mapping = VirtualNetworkInterfaceHost { virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 247, 220, 162])), physical_host_ip: fd00:1122:3344:101::1, vni: Vni(16235365) }
21082024-08-28T01:02:33.420ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 730 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:40427 req_id = ad89ecd2-62e4-4cf5-a835-de58eddd5a09 response_code = 204 uri = /v2p
21092024-08-28T01:02:33.546ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:40849
21102024-08-28T01:02:33.550ZINFOSledAgent (InstanceManager): ensuring instance is registered file = sled-agent/src/instance_manager.rs:570 hardware = InstanceHardware { properties: InstanceProperties { ncpus: InstanceCpuCount(2), memory: ByteCount(1073741824), hostname: Hostname("localshark") }, nics: [NetworkInterface { id: c475b961-fc72-4363-b146-f13036d1dd45, kind: Instance { id: d5200ea9-f81d-4e22-b797-ad91ab2270c8 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 247, 220, 162])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(16235365), primary: true, slot: 0, transit_ips: [] }], source_nat: SourceNatConfig { ip: 10.151.1.175, first_port: 0, last_port: 16383 }, ephemeral_ip: Some(10.151.1.176), floating_ips: [], firewall_rules: [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: c475b961-fc72-4363-b146-f13036d1dd45, kind: Instance { id: d5200ea9-f81d-4e22-b797-ad91ab2270c8 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 247, 220, 162])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(16235365), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: c475b961-fc72-4363-b146-f13036d1dd45, kind: Instance { id: d5200ea9-f81d-4e22-b797-ad91ab2270c8 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 247, 220, 162])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(16235365), primary: true, slot: 0, transit_ips: [] }], filter_hosts: Some([Vpc(Vni(16235365))]), filter_ports: None, filter_protocols: None, action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: c475b961-fc72-4363-b146-f13036d1dd45, kind: Instance { id: d5200ea9-f81d-4e22-b797-ad91ab2270c8 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 247, 220, 162])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(16235365), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(22), last: L4Port(22) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }], dhcp_config: DhcpConfig { dns_servers: [1.1.1.1, 9.9.9.9], host_domain: None, search_domains: [] }, disks: [DiskRequest { device: "nvme", name: "disk-6a0b3883249", read_only: false, slot: Slot(0), volume_construction_request: Volume { block_size: 512, id: 431c70a8-6da2-437a-9b24-289d6e00ea92, read_only_parent: Some(Volume { block_size: 512, id: 03629dab-ffe1-4938-972f-9ddb766ba3c5, read_only_parent: None, sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 2, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 832bb390-a013-402e-8ebd-34381ded9d70, key: Some("igIR2vULqVM84SQQ46gLEpE5X4dSkt/7G2vm2WBVdU8="), key_pem: None, lossy: false, read_only: true, root_cert_pem: None, target: ["[fd00:1122:3344:101::1a]:19001", "[fd00:1122:3344:101::14]:19001", "[fd00:1122:3344:101::19]:19002"] } }] }), sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 1, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 431c70a8-6da2-437a-9b24-289d6e00ea92, key: Some("2Fl9hckyGXn8cH+Vo5Imu7VgHyCOHXoipnJ3jjPD8cQ="), key_pem: None, lossy: false, read_only: false, root_cert_pem: None, target: ["[fd00:1122:3344:101::12]:19001", "[fd00:1122:3344:101::14]:19002", "[fd00:1122:3344:101::19]:19003"] } }] } }], cloud_init_bytes: Some(..) } instance_id = d5200ea9-f81d-4e22-b797-ad91ab2270c8 instance_runtime = InstanceRuntimeState { propolis_id: Some(a9c40859-4ce6-487f-af93-4064b962a346 (propolis)), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2024-08-28T01:02:33.105822388Z } metadata = InstanceMetadata { silo_id: b8478af8-2fe4-4899-96a9-17a68c059f6b, project_id: 1b1e2353-8423-4488-8cf9-a7ca5dd8b6f8 } propolis_addr = [fd00:1122:3344:101::1:0]:12400 propolis_id = a9c40859-4ce6-487f-af93-4064b962a346 vmm_runtime = VmmRuntimeState { state: Starting, gen: Generation(1), time_updated: 2024-08-28T01:02:33.032584Z }
21112024-08-28T01:02:33.550ZINFOSledAgent (InstanceManager): registering new instance file = sled-agent/src/instance_manager.rs:603 instance_id = d5200ea9-f81d-4e22-b797-ad91ab2270c8 propolis_id = a9c40859-4ce6-487f-af93-4064b962a346
21122024-08-28T01:02:33.550ZINFOSledAgent (InstanceManager): initializing new Instance file = sled-agent/src/instance.rs:991 instance_id = d5200ea9-f81d-4e22-b797-ad91ab2270c8 migration_id = None propolis_id = a9c40859-4ce6-487f-af93-4064b962a346 state = InstanceInitialState { hardware: InstanceHardware { properties: InstanceProperties { ncpus: InstanceCpuCount(2), memory: ByteCount(1073741824), hostname: Hostname("localshark") }, nics: [NetworkInterface { id: c475b961-fc72-4363-b146-f13036d1dd45, kind: Instance { id: d5200ea9-f81d-4e22-b797-ad91ab2270c8 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 247, 220, 162])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(16235365), primary: true, slot: 0, transit_ips: [] }], source_nat: SourceNatConfig { ip: 10.151.1.175, first_port: 0, last_port: 16383 }, ephemeral_ip: Some(10.151.1.176), floating_ips: [], firewall_rules: [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: c475b961-fc72-4363-b146-f13036d1dd45, kind: Instance { id: d5200ea9-f81d-4e22-b797-ad91ab2270c8 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 247, 220, 162])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(16235365), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: c475b961-fc72-4363-b146-f13036d1dd45, kind: Instance { id: d5200ea9-f81d-4e22-b797-ad91ab2270c8 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 247, 220, 162])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(16235365), primary: true, slot: 0, transit_ips: [] }], filter_hosts: Some([Vpc(Vni(16235365))]), filter_ports: None, filter_protocols: None, action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: c475b961-fc72-4363-b146-f13036d1dd45, kind: Instance { id: d5200ea9-f81d-4e22-b797-ad91ab2270c8 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 247, 220, 162])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(16235365), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(22), last: L4Port(22) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }], dhcp_config: DhcpConfig { dns_servers: [1.1.1.1, 9.9.9.9], host_domain: None, search_domains: [] }, disks: [DiskRequest { device: "nvme", name: "disk-6a0b3883249", read_only: false, slot: Slot(0), volume_construction_request: Volume { block_size: 512, id: 431c70a8-6da2-437a-9b24-289d6e00ea92, read_only_parent: Some(Volume { block_size: 512, id: 03629dab-ffe1-4938-972f-9ddb766ba3c5, read_only_parent: None, sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 2, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 832bb390-a013-402e-8ebd-34381ded9d70, key: Some("igIR2vULqVM84SQQ46gLEpE5X4dSkt/7G2vm2WBVdU8="), key_pem: None, lossy: false, read_only: true, root_cert_pem: None, target: ["[fd00:1122:3344:101::1a]:19001", "[fd00:1122:3344:101::14]:19001", "[fd00:1122:3344:101::19]:19002"] } }] }), sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 1, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 431c70a8-6da2-437a-9b24-289d6e00ea92, key: Some("2Fl9hckyGXn8cH+Vo5Imu7VgHyCOHXoipnJ3jjPD8cQ="), key_pem: None, lossy: false, read_only: false, root_cert_pem: None, target: ["[fd00:1122:3344:101::12]:19001", "[fd00:1122:3344:101::14]:19002", "[fd00:1122:3344:101::19]:19003"] } }] } }], cloud_init_bytes: Some(..) }, vmm_runtime: VmmRuntimeState { state: Starting, gen: Generation(1), time_updated: 2024-08-28T01:02:33.032584Z }, propolis_addr: [fd00:1122:3344:101::1:0]:12400, migration_id: None }
21132024-08-28T01:02:33.551ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 4700 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:40849 req_id = 6affdc11-d805-4c61-b369-0ac2d930e71c response_code = 200 uri = /vmms/a9c40859-4ce6-487f-af93-4064b962a346
21142024-08-28T01:02:33.655ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:38913
21152024-08-28T01:02:33.655ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 70 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:38913 req_id = f2bb1847-a14e-462f-9af6-54e506afe175 response_code = 200 uri = /vmms/a9c40859-4ce6-487f-af93-4064b962a346/state
21162024-08-28T01:02:33.657ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte6", ip: 172.30.0.5, mac: MacAddr6([168, 64, 37, 247, 220, 162]), slot: 0, vni: Vni { inner: 16235365 }, subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.0.1 } }) }
21172024-08-28T01:02:33.657ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
21182024-08-28T01:02:33.701ZINFOSledAgent (InstanceManager): Configuring new Omicron zone: oxz_propolis-server_a9c40859-4ce6-487f-af93-4064b962a346 file = illumos-utils/src/zone.rs:316 instance_id = d5200ea9-f81d-4e22-b797-ad91ab2270c8 propolis_id = a9c40859-4ce6-487f-af93-4064b962a346
21192024-08-28T01:02:33.726ZINFOSledAgent (InstanceManager): Installing Omicron zone: oxz_propolis-server_a9c40859-4ce6-487f-af93-4064b962a346 file = illumos-utils/src/zone.rs:354 instance_id = d5200ea9-f81d-4e22-b797-ad91ab2270c8 propolis_id = a9c40859-4ce6-487f-af93-4064b962a346
21202024-08-28T01:02:34.933ZINFOSledAgent (InstanceManager): Profile for oxz_propolis-server_a9c40859-4ce6-487f-af93-4064b962a346: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="system/illumos/propolis-server"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlInstance0'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::1:0'/> <propval type="astring" name="listen_port" value='12400'/> <propval type="astring" name="metric_addr" value='dns'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34 instance_id = d5200ea9-f81d-4e22-b797-ad91ab2270c8 propolis_id = a9c40859-4ce6-487f-af93-4064b962a346
21212024-08-28T01:02:34.934ZINFOSledAgent (InstanceManager): Booting oxz_propolis-server_a9c40859-4ce6-487f-af93-4064b962a346 zone file = illumos-utils/src/running_zone.rs:488 instance_id = d5200ea9-f81d-4e22-b797-ad91ab2270c8 propolis_id = a9c40859-4ce6-487f-af93-4064b962a346 zone = oxz_propolis-server_a9c40859-4ce6-487f-af93-4064b962a346
21222024-08-28T01:02:35.123ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_a9c40859-4ce6-487f-af93-4064b962a346") failed: Property not found. retry in 60.540673ms file = illumos-utils/src/svc.rs:36 instance_id = d5200ea9-f81d-4e22-b797-ad91ab2270c8 propolis_id = a9c40859-4ce6-487f-af93-4064b962a346 zone = oxz_propolis-server_a9c40859-4ce6-487f-af93-4064b962a346
21232024-08-28T01:02:35.210ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_a9c40859-4ce6-487f-af93-4064b962a346") failed: Property not found. retry in 146.244161ms file = illumos-utils/src/svc.rs:36 instance_id = d5200ea9-f81d-4e22-b797-ad91ab2270c8 propolis_id = a9c40859-4ce6-487f-af93-4064b962a346 zone = oxz_propolis-server_a9c40859-4ce6-487f-af93-4064b962a346
21242024-08-28T01:02:35.380ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_a9c40859-4ce6-487f-af93-4064b962a346") failed: Property not found. retry in 292.764055ms file = illumos-utils/src/svc.rs:36 instance_id = d5200ea9-f81d-4e22-b797-ad91ab2270c8 propolis_id = a9c40859-4ce6-487f-af93-4064b962a346 zone = oxz_propolis-server_a9c40859-4ce6-487f-af93-4064b962a346
21252024-08-28T01:02:35.689ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_a9c40859-4ce6-487f-af93-4064b962a346") failed: Property not found. retry in 326.485302ms file = illumos-utils/src/svc.rs:36 instance_id = d5200ea9-f81d-4e22-b797-ad91ab2270c8 propolis_id = a9c40859-4ce6-487f-af93-4064b962a346 zone = oxz_propolis-server_a9c40859-4ce6-487f-af93-4064b962a346
21262024-08-28T01:02:36.033ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_a9c40859-4ce6-487f-af93-4064b962a346") failed: Property not found. retry in 1.109781909s file = illumos-utils/src/svc.rs:36 instance_id = d5200ea9-f81d-4e22-b797-ad91ab2270c8 propolis_id = a9c40859-4ce6-487f-af93-4064b962a346 zone = oxz_propolis-server_a9c40859-4ce6-487f-af93-4064b962a346
21272024-08-28T01:02:36.365ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:44670
21282024-08-28T01:02:36.365ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 107 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:44670 req_id = 336a5b6e-1e49-4cd8-bddf-ea5bf6773508 response_code = 200 uri = /vpc-routes
21292024-08-28T01:02:36.381ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:52639
21302024-08-28T01:02:36.509ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:53560
21312024-08-28T01:02:36.509ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 169 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:53560 req_id = 634df0f6-e750-4ea9-857d-c7b70a60ff41 response_code = 200 uri = /v2p
21322024-08-28T01:02:36.560ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 234 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:44670 req_id = 17517c5c-fded-4929-b05f-c6fb76f602b0 response_code = 204 uri = /vpc-routes
21332024-08-28T01:02:36.567ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:45664
21342024-08-28T01:02:36.567ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
21352024-08-28T01:02:36.609ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
21362024-08-28T01:02:36.669ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 102221 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:45664 req_id = 32855e61-dca8-4c3a-8092-1b4c5cc233ad response_code = 204 uri = /switch-ports
21372024-08-28T01:02:36.670ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 123 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:45664 req_id = a7c5c8d5-6658-4416-9cb6-fb1459195356 response_code = 200 uri = /network-bootstore-config
21382024-08-28T01:02:37.072ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:63655
21392024-08-28T01:02:37.095ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:47044
21402024-08-28T01:02:37.095ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 56 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:47044 req_id = 96002864-0f0a-4b8d-ba61-795fbca37e5b response_code = 200 uri = /vpc-routes
21412024-08-28T01:02:37.137ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:58014
21422024-08-28T01:02:37.138ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 987 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:58014 req_id = 0b3de740-3de3-4f5b-8032-43820500b0fd response_code = 200 uri = /vpc-routes
21432024-08-28T01:02:37.157ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_a9c40859-4ce6-487f-af93-4064b962a346") failed: Property not found. retry in 754.753142ms file = illumos-utils/src/svc.rs:36 instance_id = d5200ea9-f81d-4e22-b797-ad91ab2270c8 propolis_id = a9c40859-4ce6-487f-af93-4064b962a346 zone = oxz_propolis-server_a9c40859-4ce6-487f-af93-4064b962a346
21442024-08-28T01:02:37.157ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:34723
21452024-08-28T01:02:37.162ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 161 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:58014 req_id = 36e93ab0-9354-4251-955d-0912ba137287 response_code = 204 uri = /vpc-routes
21462024-08-28T01:02:37.186ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:37691
21472024-08-28T01:02:37.186ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 122 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:37691 req_id = 1abe9c4a-fea1-47f0-8c06-665a1c49a554 response_code = 200 uri = /v2p
21482024-08-28T01:02:37.193ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 90 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:47044 req_id = 09788406-eb18-4f55-9562-f3223154f3cb response_code = 204 uri = /vpc-routes
21492024-08-28T01:02:37.234ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:61005
21502024-08-28T01:02:37.234ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
21512024-08-28T01:02:37.277ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
21522024-08-28T01:02:37.335ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 101355 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:61005 req_id = bd535821-16ba-4d25-8775-515fd7221e62 response_code = 204 uri = /switch-ports
21532024-08-28T01:02:37.336ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:57052
21542024-08-28T01:02:37.336ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:40853
21552024-08-28T01:02:37.337ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
21562024-08-28T01:02:37.337ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1046 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:61005 req_id = 77c2b8a4-bc2e-40e2-bd16-9a8833cce0ab response_code = 200 uri = /network-bootstore-config
21572024-08-28T01:02:37.337ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 905 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:40853 req_id = e30705f1-094c-4245-ad3e-3d82583754aa response_code = 200 uri = /v2p
21582024-08-28T01:02:37.375ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
21592024-08-28T01:02:37.431ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 95336 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:57052 req_id = 23b77d05-bab6-4ef2-ad7d-25c1a3a73406 response_code = 204 uri = /switch-ports
21602024-08-28T01:02:37.432ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 168 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:57052 req_id = e6fdeaa2-44ad-410f-9e81-f11d1316ea5e response_code = 200 uri = /network-bootstore-config
21612024-08-28T01:02:37.924ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_a9c40859-4ce6-487f-af93-4064b962a346") failed: Property not found. retry in 1.256389221s file = illumos-utils/src/svc.rs:36 instance_id = d5200ea9-f81d-4e22-b797-ad91ab2270c8 propolis_id = a9c40859-4ce6-487f-af93-4064b962a346 zone = oxz_propolis-server_a9c40859-4ce6-487f-af93-4064b962a346
21622024-08-28T01:02:39.194ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_a9c40859-4ce6-487f-af93-4064b962a346") failed: Property not found. retry in 1.084210382s file = illumos-utils/src/svc.rs:36 instance_id = d5200ea9-f81d-4e22-b797-ad91ab2270c8 propolis_id = a9c40859-4ce6-487f-af93-4064b962a346 zone = oxz_propolis-server_a9c40859-4ce6-487f-af93-4064b962a346
21632024-08-28T01:02:40.314ZINFOSledAgent (InstanceManager): Started propolis in zone: oxz_propolis-server_a9c40859-4ce6-487f-af93-4064b962a346 file = sled-agent/src/instance.rs:1449 instance_id = d5200ea9-f81d-4e22-b797-ad91ab2270c8 propolis_id = a9c40859-4ce6-487f-af93-4064b962a346
21642024-08-28T01:02:40.324ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_a9c40859-4ce6-487f-af93-4064b962a346") failed: Property not found. retry in 66.14408ms file = illumos-utils/src/svc.rs:36 instance_id = d5200ea9-f81d-4e22-b797-ad91ab2270c8 propolis_id = a9c40859-4ce6-487f-af93-4064b962a346
21652024-08-28T01:02:40.400ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_a9c40859-4ce6-487f-af93-4064b962a346") failed: Property not found. retry in 128.080953ms file = illumos-utils/src/svc.rs:36 instance_id = d5200ea9-f81d-4e22-b797-ad91ab2270c8 propolis_id = a9c40859-4ce6-487f-af93-4064b962a346
21662024-08-28T01:02:40.539ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_a9c40859-4ce6-487f-af93-4064b962a346") failed: Property not found. retry in 289.54388ms file = illumos-utils/src/svc.rs:36 instance_id = d5200ea9-f81d-4e22-b797-ad91ab2270c8 propolis_id = a9c40859-4ce6-487f-af93-4064b962a346
21672024-08-28T01:02:40.839ZINFOSledAgent (InstanceManager): Propolis SMF service is online file = sled-agent/src/instance.rs:1458 instance_id = d5200ea9-f81d-4e22-b797-ad91ab2270c8 propolis_id = a9c40859-4ce6-487f-af93-4064b962a346
21682024-08-28T01:02:40.877ZINFOSledAgent (InstanceManager): Propolis HTTP server online file = sled-agent/src/instance.rs:1482 instance_id = d5200ea9-f81d-4e22-b797-ad91ab2270c8 propolis_id = a9c40859-4ce6-487f-af93-4064b962a346
21692024-08-28T01:02:41.017ZINFOSledAgent (InstanceManager): result of instance_ensure call is Ok(InstanceEnsureResponse { migrate: None }) file = sled-agent/src/instance.rs:727 instance_id = d5200ea9-f81d-4e22-b797-ad91ab2270c8 propolis_id = a9c40859-4ce6-487f-af93-4064b962a346
21702024-08-28T01:02:41.018ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 4637238 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:52639 req_id = b4f17a22-03b1-48cd-a797-0acb5265afd8 response_code = 200 uri = /vmms/a9c40859-4ce6-487f-af93-4064b962a346/state
21712024-08-28T01:02:41.018ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3860509 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:34723 req_id = e9f95731-1197-44f3-90bc-6df42b36b670 response_code = 200 uri = /vmms/a9c40859-4ce6-487f-af93-4064b962a346/state
21722024-08-28T01:02:41.018ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3945844 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:63655 req_id = 9363c966-6220-433f-958b-16e41895f308 response_code = 200 uri = /vmms/a9c40859-4ce6-487f-af93-4064b962a346/state
21732024-08-28T01:02:41.019ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Creating), migration_in: None, migration_out: None, time: 2024-08-28T01:02:41.018843394Z } file = sled-agent/src/instance.rs:594 instance_id = d5200ea9-f81d-4e22-b797-ad91ab2270c8 propolis_id = a9c40859-4ce6-487f-af93-4064b962a346
21742024-08-28T01:02:41.019ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:616 instance_id = d5200ea9-f81d-4e22-b797-ad91ab2270c8 new_vmm_state = VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2024-08-28T01:02:41.018843394Z } propolis_id = a9c40859-4ce6-487f-af93-4064b962a346
21752024-08-28T01:02:41.019ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:514 instance_id = d5200ea9-f81d-4e22-b797-ad91ab2270c8 propolis_id = a9c40859-4ce6-487f-af93-4064b962a346 state = SledVmmState { vmm_state: VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2024-08-28T01:02:41.018843394Z }, migration_in: None, migration_out: None }
21762024-08-28T01:02:41.071ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Running), migration_in: None, migration_out: None, time: 2024-08-28T01:02:41.071505252Z } file = sled-agent/src/instance.rs:594 instance_id = d5200ea9-f81d-4e22-b797-ad91ab2270c8 propolis_id = a9c40859-4ce6-487f-af93-4064b962a346
21772024-08-28T01:02:41.071ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:616 instance_id = d5200ea9-f81d-4e22-b797-ad91ab2270c8 new_vmm_state = VmmRuntimeState { state: Running, gen: Generation(3), time_updated: 2024-08-28T01:02:41.071505252Z } propolis_id = a9c40859-4ce6-487f-af93-4064b962a346
21782024-08-28T01:02:41.071ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:514 instance_id = d5200ea9-f81d-4e22-b797-ad91ab2270c8 propolis_id = a9c40859-4ce6-487f-af93-4064b962a346 state = SledVmmState { vmm_state: VmmRuntimeState { state: Running, gen: Generation(3), time_updated: 2024-08-28T01:02:41.071505252Z }, migration_in: None, migration_out: None }
21792024-08-28T01:02:51.177ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:52362
21802024-08-28T01:02:51.178ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1091 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:52362 req_id = f1ba1ccf-e11d-4bb7-b4f6-757583708d05 response_code = 200 uri = /vmms/a9c40859-4ce6-487f-af93-4064b962a346/state
21812024-08-28T01:02:51.178ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Stopping), migration_in: None, migration_out: None, time: 2024-08-28T01:02:51.178530376Z } file = sled-agent/src/instance.rs:594 instance_id = d5200ea9-f81d-4e22-b797-ad91ab2270c8 propolis_id = a9c40859-4ce6-487f-af93-4064b962a346
21822024-08-28T01:02:51.178ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:616 instance_id = d5200ea9-f81d-4e22-b797-ad91ab2270c8 new_vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(5), time_updated: 2024-08-28T01:02:51.178530376Z } propolis_id = a9c40859-4ce6-487f-af93-4064b962a346
21832024-08-28T01:02:51.178ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:514 instance_id = d5200ea9-f81d-4e22-b797-ad91ab2270c8 propolis_id = a9c40859-4ce6-487f-af93-4064b962a346 state = SledVmmState { vmm_state: VmmRuntimeState { state: Stopping, gen: Generation(5), time_updated: 2024-08-28T01:02:51.178530376Z }, migration_in: None, migration_out: None }
21842024-08-28T01:02:51.187ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Destroyed), migration_in: None, migration_out: None, time: 2024-08-28T01:02:51.187638295Z } file = sled-agent/src/instance.rs:594 instance_id = d5200ea9-f81d-4e22-b797-ad91ab2270c8 propolis_id = a9c40859-4ce6-487f-af93-4064b962a346
21852024-08-28T01:02:51.187ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:616 instance_id = d5200ea9-f81d-4e22-b797-ad91ab2270c8 new_vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(6), time_updated: 2024-08-28T01:02:51.187638295Z } propolis_id = a9c40859-4ce6-487f-af93-4064b962a346
21862024-08-28T01:02:51.187ZINFOSledAgent (InstanceManager): terminating VMM that has exited file = sled-agent/src/instance.rs:630 instance_id = d5200ea9-f81d-4e22-b797-ad91ab2270c8 propolis_id = a9c40859-4ce6-487f-af93-4064b962a346
21872024-08-28T01:02:51.187ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
21882024-08-28T01:02:51.187ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
21892024-08-28T01:02:51.187ZINFOSledAgent (ZoneBundler): creating zone bundle context = ZoneBundleContext { storage_dirs: ["/pool/int/1d8f47b2-ceb5-4768-b5e0-de73f9c6288a/debug/bundle/zone", "/pool/int/2701509e-5560-4f63-bb12-b820fcc5e4cb/debug/bundle/zone"], cause: TerminatedInstance, extra_log_dirs: ["/pool/ext/2deff32e-1106-452e-a0e4-7f5633a4a1c4/crypt/debug", "/pool/ext/903e5bdb-4fa3-466b-9cc6-cb6c1205c20b/crypt/debug", "/pool/ext/8c0884cf-f16d-4d49-a677-c20655479168/crypt/debug", "/pool/ext/39cc409d-9820-46b7-b64f-11f4d431b612/crypt/debug", "/pool/ext/753cff17-fbdf-48d1-960a-a1fe6d8c01ed/crypt/debug", "/pool/ext/fe61a674-6b99-47f9-b238-13f6d9a31093/crypt/debug", "/pool/ext/9b7eedae-40f6-4e96-bd9d-f89d715d8ad7/crypt/debug", "/pool/ext/1a002b08-6021-4882-a443-e176f8f87821/crypt/debug", "/pool/ext/2936c097-160d-4a3f-a5e8-a1c6ffbf1edb/crypt/debug"] } file = sled-agent/src/zone_bundle.rs:339 zone_name = oxz_propolis-server_a9c40859-4ce6-487f-af93-4064b962a346
21902024-08-28T01:02:51.188ZWARNSledAgent (kstat-sampler-worker): target removed with queued samples file = oximeter/instruments/src/kstat/sampler.rs:557 id = 14216105680496823962 n_samples = 6 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
21912024-08-28T01:02:51.188ZWARNSledAgent (kstat-sampler-worker): target removed with queued samples file = oximeter/instruments/src/kstat/sampler.rs:557 id = 2706487967370437348 n_samples = 6 sled_id = 45733be1-aef6-43c8-be71-719893a1feda
21922024-08-28T01:02:51.837ZINFOSledAgent (ZoneBundler): finished zone bundle file = sled-agent/src/zone_bundle.rs:1137 metadata = ZoneBundleMetadata { id: ZoneBundleId { zone_name: "oxz_propolis-server_a9c40859-4ce6-487f-af93-4064b962a346", bundle_id: 713b0f3e-7c0d-4547-89c0-26ad036a1982 }, time_created: 2024-08-28T01:02:51.188222322Z, version: 0, cause: TerminatedInstance }
21932024-08-28T01:02:51.837ZWARNSledAgent (InstanceManager): Halting and removing zone: oxz_propolis-server_a9c40859-4ce6-487f-af93-4064b962a346 file = sled-agent/src/instance.rs:826 instance_id = d5200ea9-f81d-4e22-b797-ad91ab2270c8 propolis_id = a9c40859-4ce6-487f-af93-4064b962a346
21942024-08-28T01:02:53.601ZINFOSledAgent (InstanceManager): halt_and_remove_logged: Previous zone state: Running file = illumos-utils/src/zone.rs:270 instance_id = d5200ea9-f81d-4e22-b797-ad91ab2270c8 propolis_id = a9c40859-4ce6-487f-af93-4064b962a346
21952024-08-28T01:02:53.620ZINFOSledAgent (PortManager): Removed route set for subnet file = illumos-utils/src/opte/port_manager.rs:904 id = RouterId { vni: Vni(16235365), kind: System }
21962024-08-28T01:02:53.620ZINFOSledAgent (PortManager): Removed route set for subnet file = illumos-utils/src/opte/port_manager.rs:904 id = RouterId { vni: Vni(16235365), kind: Custom(V4(Ipv4Net { addr: 172.30.0.0, width: 22 })) }
21972024-08-28T01:02:53.622ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:514 instance_id = d5200ea9-f81d-4e22-b797-ad91ab2270c8 propolis_id = a9c40859-4ce6-487f-af93-4064b962a346 state = SledVmmState { vmm_state: VmmRuntimeState { state: Destroyed, gen: Generation(7), time_updated: 2024-08-28T01:02:53.622178466Z }, migration_in: None, migration_out: None }
21982024-08-28T01:02:53.645ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:514 instance_id = d5200ea9-f81d-4e22-b797-ad91ab2270c8 propolis_id = a9c40859-4ce6-487f-af93-4064b962a346 state = SledVmmState { vmm_state: VmmRuntimeState { state: Destroyed, gen: Generation(7), time_updated: 2024-08-28T01:02:53.622178466Z }, migration_in: None, migration_out: None }
21992024-08-28T01:02:53.648ZINFOSledAgent (InstanceManager): State monitoring task complete file = sled-agent/src/instance.rs:763 instance_id = d5200ea9-f81d-4e22-b797-ad91ab2270c8 propolis_id = a9c40859-4ce6-487f-af93-4064b962a346
22002024-08-28T01:02:54.026ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:46072
22012024-08-28T01:02:54.026ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 76 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:46072 req_id = f426fec4-65df-44c8-ba19-a91ad9ebc93a response_code = 200 uri = /vpc-routes
22022024-08-28T01:02:54.035ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 626 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:46072 req_id = ba256e74-0c3b-4a6b-a8f8-c2189f9f2f6b response_code = 204 uri = /vpc-routes
22032024-08-28T01:02:54.100ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:48601
22042024-08-28T01:02:54.100ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 137 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:48601 req_id = 1a9a140d-8226-4b3e-9fbf-264eb078927a response_code = 200 uri = /v2p
22052024-08-28T01:02:54.209ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:35482
22062024-08-28T01:02:54.209ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 154 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:35482 req_id = a49c5223-0526-41a3-8d61-17131bf29f3b response_code = 200 uri = /v2p
22072024-08-28T01:02:54.862ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:65097
22082024-08-28T01:02:54.862ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 76 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:65097 req_id = f49684e4-c2db-49cc-a463-8f29a2c68438 response_code = 200 uri = /baseboard
22092024-08-28T01:02:55.132ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:36862
22102024-08-28T01:02:55.133ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 841 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:36862 req_id = f900724b-a049-4737-81dc-ddd1b6fb9c85 response_code = 200 uri = /vpc-routes
22112024-08-28T01:02:55.141ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 127 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:36862 req_id = 096e36c8-a367-4917-a6cd-d491ab312c24 response_code = 204 uri = /vpc-routes