Max level shown:
1 [ Aug 28 03:59:46 Enabled. ]
2 [ Aug 28 03:59:46 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/sled-agent/sled-agent run /opt/oxide/sled-agent/pkg/config.toml &"). ]
3 [ Aug 28 03:59:46 Method "start" exited with status 0. ]
4 [ Aug 28 03:59:46 Rereading configuration. ]
5 [ Aug 28 03:59:46 No 'refresh' method defined. Treating as :true. ]
6 note: configured to log to "/dev/stdout"
72024-08-28T03:59:46.913ZINFOSledAgent: Ensuring contract reaper thread file = illumos-utils/src/running_zone.rs:113
82024-08-28T03:59:46.913ZINFOSledAgent (BootstrapAgentStartup): finding links ["net0", "net1"] file = sled-agent/src/bootstrap/pre_server.rs:255
92024-08-28T03:59:46.965ZINFOSledAgent (BootstrapAgentStartup): Starting mg-ddm service addr-objs = [AddrObject { interface: "net0", name: "ll" }, AddrObject { interface: "net1", name: "ll" }] file = sled-agent/src/bootstrap/pre_server.rs:262
102024-08-28T03:59:46.965ZINFOSledAgent (BootstrapAgentStartup): Importing mg-ddm service file = sled-agent/src/bootstrap/maghemite.rs:42 path = /opt/oxide/mg-ddm/pkg/ddm/manifest.xml
112024-08-28T03:59:47.079ZINFOSledAgent (BootstrapAgentStartup): Setting mg-ddm interfaces file = sled-agent/src/bootstrap/maghemite.rs:50 interfaces = ("net0/ll" "net1/ll")
122024-08-28T03:59:47.088ZINFOSledAgent (BootstrapAgentStartup): Enabling mg-ddm service file = sled-agent/src/bootstrap/maghemite.rs:56
132024-08-28T03:59:47.096ZINFOSledAgent (BootstrapAgentStartup): Ensuring zfs key directory exists file = sled-agent/src/bootstrap/pre_server.rs:272 path = /var/run/oxide/
142024-08-28T03:59:47.695ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fdb0:18c0:4d0d:a02a::, width: 64 }
152024-08-28T03:59:47.832ZINFOSledAgent: Starting KeyManager file = sled-agent/src/long_running_tasks.rs:126
162024-08-28T03:59:47.832ZINFOSledAgent: Starting StorageManager file = sled-agent/src/long_running_tasks.rs:138
172024-08-28T03:59:47.832ZINFOSledAgent: Starting StorageMonitor file = sled-agent/src/long_running_tasks.rs:151
182024-08-28T03:59:47.832ZINFOSledAgent: Starting HardwareManager file = sled-agent/src/long_running_tasks.rs:173 nongimlet_observed_disks = [] sled_mode = Scrimlet { asic: SoftNpuZone }
192024-08-28T03:59:47.832ZINFOSledAgent (DumpSetup-worker): DumpSetup poll loop started. file = sled-agent/src/dump_setup.rs:520
202024-08-28T03:59:47.832ZINFOSledAgent (HardwareManager): Creating HardwareManager file = sled-hardware/src/illumos/mod.rs:716
212024-08-28T03:59:47.835ZINFOSledAgent (HardwareManager): Generated i86pc baseboard Pc { identifier: "sock", model: "i86pc" } file = sled-hardware/src/illumos/mod.rs:623
222024-08-28T03:59:47.835ZWARNSledAgent (HardwareManager): Device is not a Gimlet (i86pc), proceeding with null hardware view file = sled-hardware/src/illumos/mod.rs:765
232024-08-28T03:59:47.835ZINFOSledAgent: Starting HardwareMonitor file = sled-agent/src/long_running_tasks.rs:187
242024-08-28T03:59:47.835ZINFOSledAgent (HardwareManager): Monitoring for hardware updates file = sled-hardware/src/illumos/mod.rs:824
252024-08-28T03:59:47.835ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/m2_0.vdev
262024-08-28T03:59:47.835ZINFOSledAgent (HardwareMonitor): Checking current full hardware snapshot disks = {} file = sled-agent/src/hardware_monitor.rs:271 underlay_network_info = None
272024-08-28T03:59:47.835ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:406
282024-08-28T03:59:47.835ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
292024-08-28T03:59:47.835ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } path = /scratch/m2_0.vdev
302024-08-28T03:59:47.846ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/m2_0.vdev file = sled-hardware/src/disk.rs:360
312024-08-28T03:59:47.846ZINFOSledAgent (StorageResources): Formatting zpool with generated ID file = sled-hardware/src/disk.rs:371 id = 51625293-e5a8-4cc4-9640-b64176e7a48e (zpool)
322024-08-28T03:59:47.969ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:307 zpool = ZpoolName { id: 51625293-e5a8-4cc4-9640-b64176e7a48e (zpool), kind: Internal }
332024-08-28T03:59:48.216ZINFOSledAgent (StorageManager): Received DetectedRawDisksChanged { raw_disks: {}, tx: ... } file = sled-storage/src/manager.rs:406
342024-08-28T03:59:48.216ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/m2_1.vdev
352024-08-28T03:59:48.216ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
362024-08-28T03:59:48.216ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
372024-08-28T03:59:48.216ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:406
382024-08-28T03:59:48.216ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
392024-08-28T03:59:48.216ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" } path = /scratch/m2_1.vdev
402024-08-28T03:59:48.225ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/m2_1.vdev file = sled-hardware/src/disk.rs:360
412024-08-28T03:59:48.225ZINFOSledAgent (StorageResources): Formatting zpool with generated ID file = sled-hardware/src/disk.rs:371 id = 4dad4c76-eb6c-4281-9f1d-a63fdb0117e1 (zpool)
422024-08-28T03:59:48.337ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:307 zpool = ZpoolName { id: 4dad4c76-eb6c-4281-9f1d-a63fdb0117e1 (zpool), kind: Internal }
432024-08-28T03:59:48.590ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_0.vdev
442024-08-28T03:59:48.590ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 51625293-e5a8-4cc4-9640-b64176e7a48e (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 4dad4c76-eb6c-4281-9f1d-a63fdb0117e1 (zpool), kind: Internal } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
452024-08-28T03:59:48.590ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
462024-08-28T03:59:48.590ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:406
472024-08-28T03:59:48.590ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
482024-08-28T03:59:48.590ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
492024-08-28T03:59:48.590ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_1.vdev
502024-08-28T03:59:48.590ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 51625293-e5a8-4cc4-9640-b64176e7a48e (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 4dad4c76-eb6c-4281-9f1d-a63fdb0117e1 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
512024-08-28T03:59:48.590ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
522024-08-28T03:59:48.590ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
532024-08-28T03:59:48.590ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:406
542024-08-28T03:59:48.590ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
552024-08-28T03:59:48.590ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_2.vdev
562024-08-28T03:59:48.590ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 51625293-e5a8-4cc4-9640-b64176e7a48e (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 4dad4c76-eb6c-4281-9f1d-a63fdb0117e1 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
572024-08-28T03:59:48.590ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
582024-08-28T03:59:48.590ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
592024-08-28T03:59:48.590ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:406
602024-08-28T03:59:48.590ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
612024-08-28T03:59:48.590ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_3.vdev
622024-08-28T03:59:48.590ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 51625293-e5a8-4cc4-9640-b64176e7a48e (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 4dad4c76-eb6c-4281-9f1d-a63fdb0117e1 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
632024-08-28T03:59:48.590ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
642024-08-28T03:59:48.590ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
652024-08-28T03:59:48.590ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:406
662024-08-28T03:59:48.590ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
672024-08-28T03:59:48.590ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_4.vdev
682024-08-28T03:59:48.590ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 51625293-e5a8-4cc4-9640-b64176e7a48e (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 4dad4c76-eb6c-4281-9f1d-a63fdb0117e1 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
692024-08-28T03:59:48.590ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
702024-08-28T03:59:48.590ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
712024-08-28T03:59:48.590ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:406
722024-08-28T03:59:48.590ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
732024-08-28T03:59:48.590ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_5.vdev
742024-08-28T03:59:48.590ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:406
752024-08-28T03:59:48.590ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
762024-08-28T03:59:48.590ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 51625293-e5a8-4cc4-9640-b64176e7a48e (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 4dad4c76-eb6c-4281-9f1d-a63fdb0117e1 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
772024-08-28T03:59:48.590ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
782024-08-28T03:59:48.590ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_6.vdev
792024-08-28T03:59:48.590ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
802024-08-28T03:59:48.590ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:406
812024-08-28T03:59:48.590ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
822024-08-28T03:59:48.590ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_7.vdev
832024-08-28T03:59:48.590ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 51625293-e5a8-4cc4-9640-b64176e7a48e (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 4dad4c76-eb6c-4281-9f1d-a63fdb0117e1 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
842024-08-28T03:59:48.590ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:406
852024-08-28T03:59:48.590ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
862024-08-28T03:59:48.590ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
872024-08-28T03:59:48.590ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
882024-08-28T03:59:48.590ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_8.vdev
892024-08-28T03:59:48.590ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 51625293-e5a8-4cc4-9640-b64176e7a48e (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 4dad4c76-eb6c-4281-9f1d-a63fdb0117e1 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
902024-08-28T03:59:48.590ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:406
912024-08-28T03:59:48.590ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
922024-08-28T03:59:48.590ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
932024-08-28T03:59:48.590ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
942024-08-28T03:59:48.590ZINFOSledAgent: Waiting for boot disk file = sled-agent/src/long_running_tasks.rs:97
952024-08-28T03:59:48.590ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 51625293-e5a8-4cc4-9640-b64176e7a48e (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 4dad4c76-eb6c-4281-9f1d-a63fdb0117e1 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
962024-08-28T03:59:48.590ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
972024-08-28T03:59:48.590ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
982024-08-28T03:59:48.590ZINFOSledAgent: Found boot disk DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } file = sled-agent/src/long_running_tasks.rs:99
992024-08-28T03:59:48.590ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
1002024-08-28T03:59:48.590ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 51625293-e5a8-4cc4-9640-b64176e7a48e (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 4dad4c76-eb6c-4281-9f1d-a63fdb0117e1 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
1012024-08-28T03:59:48.590ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
1022024-08-28T03:59:48.590ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
1032024-08-28T03:59:48.590ZINFOSledAgent: Starting Bootstore file = sled-agent/src/long_running_tasks.rs:211
1042024-08-28T03:59:48.590ZINFOSledAgent (bootstore): No ledger in /pool/int/51625293-e5a8-4cc4-9640-b64176e7a48e/cluster/bootstore-fsm-state.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1052024-08-28T03:59:48.590ZINFOSledAgent (bootstore): No ledger in /pool/int/4dad4c76-eb6c-4281-9f1d-a63fdb0117e1/cluster/bootstore-fsm-state.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1062024-08-28T03:59:48.590ZINFOSledAgent (bootstore): No ledger found. Loading Fsm::State as Uninitialized file = bootstore/src/schemes/v0/storage.rs:83 peer_id = pc-sock-i86pc
1072024-08-28T03:59:48.590ZINFOSledAgent (bootstore): No ledger in /pool/int/51625293-e5a8-4cc4-9640-b64176e7a48e/cluster/bootstore-network-config.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1082024-08-28T03:59:48.590ZINFOSledAgent (bootstore): No ledger in /pool/int/4dad4c76-eb6c-4281-9f1d-a63fdb0117e1/cluster/bootstore-network-config.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1092024-08-28T03:59:48.590ZINFOSledAgent (bootstore): No ledger found for network config file = bootstore/src/schemes/v0/storage.rs:146 peer_id = pc-sock-i86pc
1102024-08-28T03:59:48.590ZINFOSledAgent: Starting Bootstore DDMD poller file = sled-agent/src/long_running_tasks.rs:216
1112024-08-28T03:59:48.590ZINFOSledAgent: Starting ZoneBundler related tasks file = sled-agent/src/long_running_tasks.rs:231
1122024-08-28T03:59:48.601ZINFOSledAgent (ServiceManager): Creating ServiceManager file = sled-agent/src/services.rs:731
1132024-08-28T03:59:48.601ZINFOSledAgent (auto-cleanup-task): top of bundle cleanup loop file = sled-agent/src/zone_bundle.rs:186 next_cleanup = Instant { tv_sec: 1275, tv_nsec: 345460165 } time_to_next_cleanup = 599.999936879s
1142024-08-28T03:59:48.601ZINFOSledAgent (ServiceManager): Ensuring scrimlet services (enabling services) file = sled-agent/src/services.rs:3861
1152024-08-28T03:59:48.601ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
1162024-08-28T03:59:48.601ZINFOSledAgent (BootstrapAgentStartup): No ledger in /pool/int/51625293-e5a8-4cc4-9640-b64176e7a48e/config/sled-agent-request.json file = common/src/ledger.rs:188
1172024-08-28T03:59:48.601ZINFOSledAgent (BootstrapAgentStartup): No ledger in /pool/int/4dad4c76-eb6c-4281-9f1d-a63fdb0117e1/config/sled-agent-request.json file = common/src/ledger.rs:188
1182024-08-28T03:59:48.604ZINFOSledAgent (dropshot (BootstrapAgent)): listening file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:205 local_addr = [fdb0:18c0:4d0d:a02a::1]:80
1192024-08-28T03:59:48.604ZINFOSledAgent (SledAgentSprocketsServer): Started listening file = sled-agent/src/bootstrap/sprockets_server.rs:44 local_addr = [fdb0:18c0:4d0d:a02a::1]:12346
1202024-08-28T03:59:48.605ZINFOSledAgent (RSS): Injecting RSS configuration: RackInitializeRequest { trust_quorum_peers: None, bootstrap_discovery: OnlyOurs, ntp_servers: [ "0.pool.ntp.org", ], dns_servers: [ 1.1.1.1, 9.9.9.9, ], internal_services_ip_pool_ranges: [ V4( Ipv4Range { first: 10.151.2.164, last: 10.151.2.173, }, ), ], external_dns_ips: [ 10.151.2.164, 10.151.2.165, ], external_dns_zone_name: "oxide.test", external_certificates: "<redacted>", recovery_silo: RecoverySiloConfig { silo_name: Name( "recovery", ), user_name: UserId( "recovery", ), user_password_hash: NewPasswordHash( PasswordHashString { string: "$argon2id$v=19$m=98304,t=13,p=1$RUlWc0ZxaHo0WFdrN0N6ZQ$S8p52j85GPvMhR/ek3GL0el/oProgTwWpHJZ8lsQQoY", encoding: B64, }, ), }, rack_network_config: RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [ PortConfigV2 { routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), nexthop: 10.151.2.100, vlan_id: None, local_pref: None, }, ], addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.2.174, width: 24, }, ), vlan_id: None, }, ], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: None, bgp_peers: [], autoneg: false, lldp: None, }, ], bgp: [], bfd: [], }, allowed_source_ips: Any, } file = sled-agent/src/rack_setup/service.rs:1053
1212024-08-28T03:59:48.605ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = internal-dns/src/resolver.rs:73
1222024-08-28T03:59:48.605ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
1232024-08-28T03:59:48.605ZINFOSledAgent (RSS): No ledger in /pool/int/51625293-e5a8-4cc4-9640-b64176e7a48e/config/rss-plan-completed.marker file = common/src/ledger.rs:188
1242024-08-28T03:59:48.605ZINFOSledAgent (RSS): No ledger in /pool/int/4dad4c76-eb6c-4281-9f1d-a63fdb0117e1/config/rss-plan-completed.marker file = common/src/ledger.rs:188
1252024-08-28T03:59:48.605ZINFOSledAgent (RSS): RSS configuration has not been fully applied yet file = sled-agent/src/rack_setup/service.rs:1113
1262024-08-28T03:59:48.605ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
1272024-08-28T03:59:48.605ZINFOSledAgent (RSS): No ledger in /pool/int/51625293-e5a8-4cc4-9640-b64176e7a48e/config/rss-sled-plan.json file = common/src/ledger.rs:188
1282024-08-28T03:59:48.605ZINFOSledAgent (RSS): No ledger in /pool/int/4dad4c76-eb6c-4281-9f1d-a63fdb0117e1/config/rss-sled-plan.json file = common/src/ledger.rs:188
1292024-08-28T03:59:48.605ZINFOSledAgent (RSS): Creating new allocation plan file = sled-agent/src/rack_setup/service.rs:1154
1302024-08-28T03:59:48.605ZINFOSledAgent (RSS): Serializing plan file = sled-agent/src/rack_setup/plan/sled.rs:155
1312024-08-28T03:59:48.605ZINFOSledAgent (RSS): Creating plan for the sled at fdb0:18c0:4d0d:a02a::1 file = sled-agent/src/rack_setup/plan/sled.rs:128
1322024-08-28T03:59:48.605ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
1332024-08-28T03:59:48.605ZINFOSledAgent (RSS): Writing ledger to /pool/int/51625293-e5a8-4cc4-9640-b64176e7a48e/config/.rss-sled-plan.json.tmp file = common/src/ledger.rs:199
1342024-08-28T03:59:48.605ZINFOSledAgent (RSS): Writing ledger to /pool/int/4dad4c76-eb6c-4281-9f1d-a63fdb0117e1/config/.rss-sled-plan.json.tmp file = common/src/ledger.rs:199
1352024-08-28T03:59:48.605ZINFOSledAgent (RSS): Sled plan written to storage: Plan { rack_id: 8b05f6f6-ff29-4d8c-aa09-514d34e6a880, sleds: { [fdb0:18c0:4d0d:a02a::1]:12346: StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: f62e5a13-51f3-4379-acf3-2f8e8601ac1f, rack_id: 8b05f6f6-ff29-4d8c-aa09-514d34e6a880, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64, }, }, }, }, }, config: RackInitializeRequest { trust_quorum_peers: None, bootstrap_discovery: OnlyOurs, ntp_servers: [ "0.pool.ntp.org", ], dns_servers: [ 1.1.1.1, 9.9.9.9, ], internal_services_ip_pool_ranges: [ V4( Ipv4Range { first: 10.151.2.164, last: 10.151.2.173, }, ), ], external_dns_ips: [ 10.151.2.164, 10.151.2.165, ], external_dns_zone_name: "oxide.test", external_certificates: "<redacted>", recovery_silo: RecoverySiloConfig { silo_name: Name( "recovery", ), user_name: UserId( "recovery", ), user_password_hash: NewPasswordHash( PasswordHashString { string: "$argon2id$v=19$m=98304,t=13,p=1$RUlWc0ZxaHo0WFdrN0N6ZQ$S8p52j85GPvMhR/ek3GL0el/oProgTwWpHJZ8lsQQoY", encoding: B64, }, ), }, rack_network_config: RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [ PortConfigV2 { routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), nexthop: 10.151.2.100, vlan_id: None, local_pref: None, }, ], addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.2.174, width: 24, }, ), vlan_id: None, }, ], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: None, bgp_peers: [], autoneg: false, lldp: None, }, ], bgp: [], bfd: [], }, allowed_source_ips: Any, }, } file = sled-agent/src/rack_setup/plan/sled.rs:175
1362024-08-28T03:59:48.605ZINFOSledAgent (RSS): Writing Rack Network Configuration to bootstore file = sled-agent/src/rack_setup/service.rs:1187
1372024-08-28T03:59:48.606ZINFOSledAgent (bootstore): Attempting to update network config with generation: 1, current_generation: 0 file = bootstore/src/schemes/v0/peer.rs:564 peer_id = pc-sock-i86pc
1382024-08-28T03:59:48.606ZINFOSledAgent (bootstore): Writing ledger to /pool/int/51625293-e5a8-4cc4-9640-b64176e7a48e/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
1392024-08-28T03:59:48.606ZINFOSledAgent (bootstore): Writing ledger to /pool/int/4dad4c76-eb6c-4281-9f1d-a63fdb0117e1/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
1402024-08-28T03:59:48.606ZINFOSledAgent (bootstore): Broadcasting network config with generation 1 file = bootstore/src/schemes/v0/peer.rs:648 peer_id = pc-sock-i86pc
1412024-08-28T03:59:48.606ZINFOSledAgent (BootstrapAgentRssHandler): Received initialization request from RSS file = sled-agent/src/bootstrap/rss_handle.rs:217 request = StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: f62e5a13-51f3-4379-acf3-2f8e8601ac1f, rack_id: 8b05f6f6-ff29-4d8c-aa09-514d34e6a880, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64 } } } } target_sled = [fdb0:18c0:4d0d:a02a::1]:12346
1422024-08-28T03:59:48.606ZINFOSledAgent (SledAgentSprocketsServer): Accepted connection file = sled-agent/src/bootstrap/sprockets_server.rs:66 remote_addr = [fdb0:18c0:4d0d:a02a::1]:35091
1432024-08-28T03:59:48.606ZINFOSledAgent (SledAgentMain): Loading Sled Agent: StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: f62e5a13-51f3-4379-acf3-2f8e8601ac1f, rack_id: 8b05f6f6-ff29-4d8c-aa09-514d34e6a880, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64 } } } } file = sled-agent/src/bootstrap/server.rs:359
1442024-08-28T03:59:48.606ZINFOSledAgent (SledAgentMain): KeyManager: using hardcoded secret retriever file = sled-agent/src/bootstrap/server.rs:374
1452024-08-28T03:59:48.606ZINFOSledAgent: setting up sled agent server file = sled-agent/src/server.rs:44
1462024-08-28T03:59:48.606ZINFOSledAgent (StorageManager): Received KeyManagerReady file = sled-storage/src/manager.rs:406
1472024-08-28T03:59:48.606ZINFOSledAgent (StorageManager): Loading storage config file = sled-storage/src/manager.rs:574
1482024-08-28T03:59:48.606ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = internal-dns/src/resolver.rs:73
1492024-08-28T03:59:48.606ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:101::, width: 64 }
1502024-08-28T03:59:48.606ZINFOSledAgent (StorageManager): No ledger in /pool/int/51625293-e5a8-4cc4-9640-b64176e7a48e/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = load_ledger
1512024-08-28T03:59:48.606ZINFOSledAgent (StorageManager): No ledger in /pool/int/4dad4c76-eb6c-4281-9f1d-a63fdb0117e1/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = load_ledger
1522024-08-28T03:59:48.606ZINFOSledAgent (StorageManager): No ledger of physical disks exists file = sled-storage/src/manager.rs:563
1532024-08-28T03:59:48.606ZINFOSledAgent (StorageManager): KeyManager ready, but no ledger detected file = sled-storage/src/manager.rs:591
1542024-08-28T03:59:48.606ZINFOSledAgent (StorageManager): automatically managing disks file = sled-storage/src/manager.rs:395
1552024-08-28T03:59:48.606ZINFOSledAgent (StorageResources): Synchronizing disk managment file = sled-storage/src/resources.rs:302
1562024-08-28T03:59:48.606ZINFOSledAgent (StorageManager): Successfully synchronized disks without error file = sled-storage/src/manager.rs:481 result = DisksManagementResult { status: [] }
1572024-08-28T03:59:48.609ZINFOSledAgent (ServiceManager): Enabling switch zone (new) file = sled-agent/src/services.rs:4158
1582024-08-28T03:59:48.609ZINFOSledAgent (ServiceManager): Starting switch zone file = sled-agent/src/services.rs:4540
1592024-08-28T03:59:48.638ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
1602024-08-28T03:59:48.656ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_switch file = illumos-utils/src/zone.rs:316
1612024-08-28T03:59:48.672ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_switch file = illumos-utils/src/zone.rs:354
1622024-08-28T03:59:48.686ZINFOSledAgent: SledAgent::new(..) starting file = sled-agent/src/sled_agent.rs:385 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f
1632024-08-28T03:59:48.686ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
1642024-08-28T03:59:48.686ZINFOSledAgent: Requested swap device of size 64 GiB file = sled-agent/src/sled_agent.rs:397 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f
1652024-08-28T03:59:48.686ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1662024-08-28T03:59:48.689ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 96.158518ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "d73d7210-0132-4faa-a0e8-05218d43d531", "content-length": "124", "date": "Wed, 28 Aug 2024 03:59:47 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "d73d7210-0132-4faa-a0e8-05218d43d531" } file = clients/ddm-admin-client/src/lib.rs:177
1672024-08-28T03:59:48.701ZINFOSledAgent: attempting to create encrypted zvol: name="oxi_51625293-e5a8-4cc4-9640-b64176e7a48e/swap", size_gb=64 file = sled-agent/src/swap_device.rs:153
1682024-08-28T03:59:48.733ZINFOSledAgent: successfully created encrypted zvol: name="oxi_51625293-e5a8-4cc4-9640-b64176e7a48e/swap", size_gb=64 file = sled-agent/src/swap_device.rs:249
1692024-08-28T03:59:48.733ZINFOSledAgent: adding swap device: swapname="/dev/zvol/dsk/oxi_51625293-e5a8-4cc4-9640-b64176e7a48e/swap" file = sled-agent/src/swap_device.rs:96
1702024-08-28T03:59:48.738ZINFOSledAgent: Mounting backing filesystems file = sled-agent/src/sled_agent.rs:412 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f
1712024-08-28T03:59:48.738ZINFOSledAgent (BackingFs): Processing fmd file = sled-agent/src/backing_fs.rs:126
1722024-08-28T03:59:48.738ZINFOSledAgent (BackingFs): Ensuring dataset oxi_51625293-e5a8-4cc4-9640-b64176e7a48e/backing/fmd file = sled-agent/src/backing_fs.rs:136
1732024-08-28T03:59:48.786ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1742024-08-28T03:59:48.787ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 173.973072ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "ff6253e7-0810-4904-b73a-4af9bd6d065a", "content-length": "124", "date": "Wed, 28 Aug 2024 03:59:48 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "ff6253e7-0810-4904-b73a-4af9bd6d065a" } file = clients/ddm-admin-client/src/lib.rs:177
1752024-08-28T03:59:48.818ZINFOSledAgent (BackingFs): Stopping service svc:/system/fmd:default file = sled-agent/src/backing_fs.rs:170
1762024-08-28T03:59:48.962ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1772024-08-28T03:59:48.962ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 178.643105ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "57e21f45-8fa2-41af-ad52-b0e9761f09cc", "content-length": "124", "date": "Wed, 28 Aug 2024 03:59:48 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "57e21f45-8fa2-41af-ad52-b0e9761f09cc" } file = clients/ddm-admin-client/src/lib.rs:177
1782024-08-28T03:59:49.037ZINFOSledAgent (BackingFs): Mounting oxi_51625293-e5a8-4cc4-9640-b64176e7a48e/backing/fmd on /var/fm/fmd file = sled-agent/src/backing_fs.rs:178
1792024-08-28T03:59:49.050ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/rsrc file = sled-agent/src/backing_fs.rs:186
1802024-08-28T03:59:49.050ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/ckpt file = sled-agent/src/backing_fs.rs:186
1812024-08-28T03:59:49.050ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/xprt file = sled-agent/src/backing_fs.rs:186
1822024-08-28T03:59:49.050ZINFOSledAgent (BackingFs): Starting service svc:/system/fmd:default file = sled-agent/src/backing_fs.rs:192
1832024-08-28T03:59:49.144ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1842024-08-28T03:59:49.145ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 241.559641ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "196d6f89-55e8-4510-9ce4-56d80520c5ac", "content-length": "124", "date": "Wed, 28 Aug 2024 03:59:48 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "196d6f89-55e8-4510-9ce4-56d80520c5ac" } file = clients/ddm-admin-client/src/lib.rs:177
1852024-08-28T03:59:49.388ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1862024-08-28T03:59:49.389ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 126.57411ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "30c738af-7065-4e16-9c3b-0d938f40e756", "content-length": "124", "date": "Wed, 28 Aug 2024 03:59:48 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "30c738af-7065-4e16-9c3b-0d938f40e756" } file = clients/ddm-admin-client/src/lib.rs:177
1872024-08-28T03:59:49.517ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1882024-08-28T03:59:49.518ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 229.136113ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "22df99fe-bac8-45c5-9de1-1e1794803986", "content-length": "124", "date": "Wed, 28 Aug 2024 03:59:48 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "22df99fe-bac8-45c5-9de1-1e1794803986" } file = clients/ddm-admin-client/src/lib.rs:177
1892024-08-28T03:59:49.748ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1902024-08-28T03:59:49.789ZINFOSledAgent: using '[AddrObject { interface: "net0", name: "ll" }, AddrObject { interface: "net1", name: "ll" }]' as data links for xde driver file = illumos-utils/src/opte/illumos.rs:90 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f
1912024-08-28T03:59:49.794ZINFOSledAgent (dropshot): listening file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:205 local_addr = [fd00:1122:3344:101::1]:60627 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f
1922024-08-28T03:59:49.794ZINFOSledAgent (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = internal-dns/src/resolver.rs:73 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f
1932024-08-28T03:59:49.795ZINFOSledAgent (producer-server): starting oximeter metric producer server address = [fd00:1122:3344:101::1]:60627 file = oximeter/producer/src/lib.rs:283 interval = 30s producer_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f
1942024-08-28T03:59:49.796ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 243.323188ms error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f
1952024-08-28T03:59:49.797ZINFOSledAgent (VmmReservoirManager): 50% of 102952026112 physical ram = 51476013056 bytes) file = sled-agent/src/vmm_reservoir.rs:267 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f
1962024-08-28T03:59:49.797ZINFOSledAgent (VmmReservoirManager): Setting reservoir size to 49090 MiB bytes file = sled-agent/src/vmm_reservoir.rs:275 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f
1972024-08-28T03:59:49.797ZINFOSledAgent (ServiceManager): sled agent started file = sled-agent/src/services.rs:915 underlay_address = fd00:1122:3344:101::1
1982024-08-28T03:59:49.797ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = internal-dns/src/resolver.rs:73
1992024-08-28T03:59:50.040ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 629.845295ms error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f
2002024-08-28T03:59:50.672ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 1.219044812s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f
2012024-08-28T03:59:51.893ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 1.511413635s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f
2022024-08-28T03:59:52.108ZINFOSledAgent (ServiceManager): Setting up dendrite service file = sled-agent/src/services.rs:2634
2032024-08-28T03:59:52.108ZINFOSledAgent (ServiceManager): Setting up lldpd service file = sled-agent/src/services.rs:2852
2042024-08-28T03:59:52.108ZINFOSledAgent (ServiceManager): Setting up MGS service file = sled-agent/src/services.rs:2504
2052024-08-28T03:59:52.108ZINFOSledAgent (ServiceManager): Setting up wicketd service file = sled-agent/src/services.rs:2551
2062024-08-28T03:59:52.108ZINFOSledAgent (ServiceManager): Setting up mgd service file = sled-agent/src/services.rs:2933
2072024-08-28T03:59:52.108ZINFOSledAgent (ServiceManager): Setting up mg-ddm service file = sled-agent/src/services.rs:2975
2082024-08-28T03:59:52.110ZINFOSledAgent (ServiceManager): Setting up tfport service file = sled-agent/src/services.rs:2777
2092024-08-28T03:59:52.111ZINFOSledAgent (ServiceManager): Setting up Simulated SP service file = sled-agent/src/services.rs:2545
2102024-08-28T03:59:52.111ZINFOSledAgent (ServiceManager): Profile for oxz_switch: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService0'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/mgs"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="address" value='[::1]:12225'/> <propval type="astring" name="id" value='0e1d8d78-006e-4d11-afc1-f62f97ae2dc4'/> <propval type="astring" name="rack_id" value='8b05f6f6-ff29-4d8c-aa09-514d34e6a880'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/wicketd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="address" value='[::1]:12226'/> <propval type="astring" name="artifact-address" value='[fdb0:18c0:4d0d:a02a::2]:12227'/> <propval type="astring" name="baseboard-file" value='/opt/oxide/baseboard.json'/> <propval type="astring" name="mgs-address" value='[::1]:12225'/> <propval type="astring" name="nexus-proxy-address" value='[::]:12229'/> <propval type="astring" name="rack-subnet" value='fd00:1122:3344::'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/switch_zone_setup"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="gz_local_link_addr" value='fe80::8:20ff:fed8:ebec'/> <property type="astring" name="link_local_links"> <astring_list> <value_node value="tfportrear0_0" /> <value_node value="oxBootstrap0" /> </astring_list> </property> <propval type="astring" name="bootstrap_addr" value='fdb0:18c0:4d0d:a02a::2'/> <propval type="astring" name="bootstrap_vnic" value='oxBootstrap0'/> <propval type="astring" name="baseboard_info" value='{"type":"pc","identifier":"sock","model":"i86pc"}'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/dendrite"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="sled_id" value='f62e5a13-51f3-4379-acf3-2f8e8601ac1f'/> <propval type="astring" name="rack_id" value='8b05f6f6-ff29-4d8c-aa09-514d34e6a880'/> <propval type="astring" name="address" value='[::1]:12224'/> <propval type="astring" name="front_ports" value='1'/> <propval type="astring" name="rear_ports" value='1'/> <propval type="astring" name="port_config" value='/opt/oxide/dendrite/misc/softnpu_single_sled_config.toml'/> <propval type="astring" name="mgmt" value='uds'/> <propval type="astring" name="uds_path" value='/opt/softnpu/stuff'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/tfport"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="host" value='[::1]'/> <propval type="astring" name="port" value='12224'/> <propval type="astring" name="flags" value='--sync-only'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/lldpd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="board_rev" value='softnpu_front_1_rear_1'/> <propval type="astring" name="scrimlet_id" value='sock'/> <propval type="astring" name="scrimlet_model" value='i86pc'/> <propval type="astring" name="address" value='[::1]:12230'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/pumpkind"> </service> <service version="1" type="service" name="oxide/mgd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="sled_uuid" value='f62e5a13-51f3-4379-acf3-2f8e8601ac1f'/> <propval type="astring" name="rack_uuid" value='8b05f6f6-ff29-4d8c-aa09-514d34e6a880'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/mg-ddm"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="mode" value='transit'/> <propval type="astring" name="dendrite" value='true'/> <propval type="astring" name="sled_uuid" value='f62e5a13-51f3-4379-acf3-2f8e8601ac1f'/> <propval type="astring" name="rack_uuid" value='8b05f6f6-ff29-4d8c-aa09-514d34e6a880'/> <propval type="astring" name="interfaces" value='tfportrear0_0/ll'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/uplink"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
2112024-08-28T03:59:52.111ZINFOSledAgent (ServiceManager): Booting oxz_switch zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_switch
2122024-08-28T03:59:52.376ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 72.291471ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2132024-08-28T03:59:52.572ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 64.662274ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2142024-08-28T03:59:52.754ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 155.691762ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2152024-08-28T03:59:52.933ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 281.029166ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2162024-08-28T03:59:53.291ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 884.108054ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2172024-08-28T03:59:53.407ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 4.73158503s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f
2182024-08-28T03:59:54.217ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 1.257022361s file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2192024-08-28T03:59:55.693ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 704.417044ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2202024-08-28T03:59:56.565ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 913.536946ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2212024-08-28T03:59:57.519ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 1.407130485s file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2222024-08-28T03:59:58.141ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 5.617331095s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f
2232024-08-28T03:59:59.205ZINFOSledAgent (dropshot (SledAgent)): listening file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:205 local_addr = [fd00:1122:3344:101::1]:12345
2242024-08-28T03:59:59.205ZINFOSledAgent (SledAgentMain): Sled Agent loaded; recording configuration file = sled-agent/src/bootstrap/server.rs:429
2252024-08-28T03:59:59.205ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
2262024-08-28T03:59:59.205ZINFOSledAgent (SledAgentMain): Writing ledger to /pool/int/51625293-e5a8-4cc4-9640-b64176e7a48e/config/.sled-agent-request.json.tmp file = common/src/ledger.rs:199
2272024-08-28T03:59:59.205ZINFOSledAgent (SledAgentMain): Writing ledger to /pool/int/4dad4c76-eb6c-4281-9f1d-a63fdb0117e1/config/.sled-agent-request.json.tmp file = common/src/ledger.rs:199
2282024-08-28T03:59:59.205ZINFOSledAgent (HardwareMonitor): Sled Agent Started file = sled-agent/src/hardware_monitor.rs:138
2292024-08-28T03:59:59.205ZINFOSledAgent (HardwareMonitor): Checking current full hardware snapshot disks = {} file = sled-agent/src/hardware_monitor.rs:271 underlay_network_info = Some((fd00:1122:3344:101::2, Some(RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56 }, infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [PortConfigV2 { routes: [RouteConfig { destination: V4(Ipv4Net { addr: 0.0.0.0, width: 0 }), nexthop: 10.151.2.100, vlan_id: None, local_pref: None }], addresses: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: None, bgp_peers: [], autoneg: false, lldp: None }], bgp: [], bfd: [] })))
2302024-08-28T03:59:59.205ZINFOSledAgent (ServiceManager): Ensuring scrimlet services (enabling services) file = sled-agent/src/services.rs:3861
2312024-08-28T03:59:59.205ZINFOSledAgent (SledAgentSprocketsServer): Connection closed file = sled-agent/src/bootstrap/sprockets_server.rs:73 remote_addr = [fdb0:18c0:4d0d:a02a::1]:35091
2322024-08-28T03:59:59.205ZINFOSledAgent (BootstrapAgentRssHandler): Peer agent initialized file = sled-agent/src/bootstrap/rss_handle.rs:108 peer = [fdb0:18c0:4d0d:a02a::1]:12346
2332024-08-28T03:59:59.205ZINFOSledAgent (BootstrapAgentRssHandler): Initialized sled agent file = sled-agent/src/bootstrap/rss_handle.rs:232 target_sled = [fdb0:18c0:4d0d:a02a::1]:12346
2342024-08-28T03:59:59.428ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
2352024-08-28T03:59:59.428ZINFOSledAgent (RSS): No ledger in /pool/int/51625293-e5a8-4cc4-9640-b64176e7a48e/config/rss-service-plan-v4.json file = common/src/ledger.rs:188
2362024-08-28T03:59:59.428ZINFOSledAgent (RSS): No ledger in /pool/int/4dad4c76-eb6c-4281-9f1d-a63fdb0117e1/config/rss-service-plan-v4.json file = common/src/ledger.rs:188
2372024-08-28T03:59:59.428ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
2382024-08-28T03:59:59.428ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
2392024-08-28T03:59:59.428ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
2402024-08-28T03:59:59.433ZINFOSledAgent (ServiceManager): Re-enabling running switch zone (new address) file = sled-agent/src/services.rs:4182 new = [fd00:1122:3344:101::2, ::1] old = [::1]
2412024-08-28T03:59:59.433ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::2 exists file = sled-agent/src/services.rs:4198
2422024-08-28T03:59:59.433ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:101::2, prefix: 64 })) file = illumos-utils/src/running_zone.rs:534 zone = oxz_switch
2432024-08-28T03:59:59.655ZINFOSledAgent (VmmReservoirManager): Finished setting reservoir size to 49090 MiB bytes file = sled-agent/src/vmm_reservoir.rs:279 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f
2442024-08-28T03:59:59.684ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:36693
2452024-08-28T03:59:59.685ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
2462024-08-28T03:59:59.685ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 226 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:36693 req_id = 39b7c5a8-08a4-4a8c-8f89-ebdcd473bd63 response_code = 200 uri = /inventory
2472024-08-28T03:59:59.715ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:54543
2482024-08-28T03:59:59.715ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 69 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:54543 req_id = 14c7bf4d-4623-4aab-b30f-c60b59d09cab response_code = 200 uri = /sled-role
2492024-08-28T03:59:59.716ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
2502024-08-28T03:59:59.716ZINFOSledAgent (RSS): Writing ledger to /pool/int/51625293-e5a8-4cc4-9640-b64176e7a48e/config/.rss-service-plan-v4.json.tmp file = common/src/ledger.rs:199
2512024-08-28T03:59:59.716ZINFOSledAgent (RSS): Writing ledger to /pool/int/4dad4c76-eb6c-4281-9f1d-a63fdb0117e1/config/.rss-service-plan-v4.json.tmp file = common/src/ledger.rs:199
2522024-08-28T03:59:59.716ZINFOSledAgent (RSS): Service plan written to storage file = sled-agent/src/rack_setup/plan/service.rs:998
2532024-08-28T03:59:59.748ZINFOSledAgent (RSS): attempting to set up sled's storage: OmicronPhysicalDisksConfig { generation: Generation(1), disks: [OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, id: ddad7798-b711-4a2c-b699-d85c5bf35100, pool_id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, id: 7c93907b-7c58-460d-818e-0ab84547fbad, pool_id: 0f906b25-a03c-4334-807e-07a343a51361 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, id: 91fa6a08-4505-48e2-b5ca-e9d43b2cfd3b, pool_id: d6170dc9-a1d4-44a9-ba24-00c07a403b1e (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, id: 848e2ed8-0b89-4355-beee-234dd6198a39, pool_id: 8955d8a9-1b85-47ea-bd5c-275ef6dc0b1a (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, id: 67af7c20-e426-4b0e-bd99-c1f8e00a1155, pool_id: fb7c10cb-4fd0-49dd-8c21-f83feafbd7dc (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, id: e6f0eebf-c766-40d8-8cad-27901b39feb6, pool_id: ba3c519e-f98a-4ee2-a345-a21845f5f7a4 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, id: 2872258c-56a1-4a3d-84ca-fdc53383ae52, pool_id: 0913dfa8-99f2-4518-8d37-827ea32c30ba (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, id: ed833b22-7b6d-4faa-a6f2-fa2abe2ed3fe, pool_id: ed7027bd-3cf9-4764-bc15-404c842ab02a (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, id: 3419f35d-cfa2-4d4b-b68c-ed68c24cfb91, pool_id: e3fa46c7-4f3a-45f7-afd8-b7799f191587 (zpool) }] } file = sled-agent/src/rack_setup/service.rs:355 sled_address = [fd00:1122:3344:101::1]:12345
2542024-08-28T03:59:59.748ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:65366
2552024-08-28T03:59:59.749ZINFOSledAgent: physical disks ensure file = sled-agent/src/sled_agent.rs:830 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f
2562024-08-28T03:59:59.749ZINFOSledAgent (StorageManager): Received OmicronPhysicalDisksEnsure { config: OmicronPhysicalDisksConfig { generation: Generation(1), disks: [OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, id: ddad7798-b711-4a2c-b699-d85c5bf35100, pool_id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, id: 7c93907b-7c58-460d-818e-0ab84547fbad, pool_id: 0f906b25-a03c-4334-807e-07a343a51361 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, id: 91fa6a08-4505-48e2-b5ca-e9d43b2cfd3b, pool_id: d6170dc9-a1d4-44a9-ba24-00c07a403b1e (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, id: 848e2ed8-0b89-4355-beee-234dd6198a39, pool_id: 8955d8a9-1b85-47ea-bd5c-275ef6dc0b1a (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, id: 67af7c20-e426-4b0e-bd99-c1f8e00a1155, pool_id: fb7c10cb-4fd0-49dd-8c21-f83feafbd7dc (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, id: e6f0eebf-c766-40d8-8cad-27901b39feb6, pool_id: ba3c519e-f98a-4ee2-a345-a21845f5f7a4 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, id: 2872258c-56a1-4a3d-84ca-fdc53383ae52, pool_id: 0913dfa8-99f2-4518-8d37-827ea32c30ba (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, id: ed833b22-7b6d-4faa-a6f2-fa2abe2ed3fe, pool_id: ed7027bd-3cf9-4764-bc15-404c842ab02a (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, id: 3419f35d-cfa2-4d4b-b68c-ed68c24cfb91, pool_id: e3fa46c7-4f3a-45f7-afd8-b7799f191587 (zpool) }] }, tx: ... } file = sled-storage/src/manager.rs:406
2572024-08-28T03:59:59.749ZINFOSledAgent (StorageManager): No ledger in /pool/int/51625293-e5a8-4cc4-9640-b64176e7a48e/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = omicron_physical_disks_ensure
2582024-08-28T03:59:59.749ZINFOSledAgent (StorageManager): No ledger in /pool/int/4dad4c76-eb6c-4281-9f1d-a63fdb0117e1/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = omicron_physical_disks_ensure
2592024-08-28T03:59:59.749ZINFOSledAgent (StorageManager): No previously-stored 'requested disks', creating new ledger file = sled-storage/src/manager.rs:645 request = omicron_physical_disks_ensure
2602024-08-28T03:59:59.749ZINFOSledAgent (StorageResources): Synchronizing disk managment file = sled-storage/src/resources.rs:302
2612024-08-28T03:59:59.749ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:342
2622024-08-28T03:59:59.749ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
2632024-08-28T03:59:59.749ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } path = /scratch/u2_0.vdev
2642024-08-28T03:59:59.760ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_0.vdev file = sled-hardware/src/disk.rs:360
2652024-08-28T03:59:59.760ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool)
2662024-08-28T03:59:59.873ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:307 zpool = ZpoolName { id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool), kind: External }
2672024-08-28T03:59:59.881ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:351
2682024-08-28T03:59:59.881ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:353
2692024-08-28T03:59:59.881ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:357
2702024-08-28T03:59:59.881ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:359
2712024-08-28T03:59:59.881ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key")
2722024-08-28T03:59:59.881ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key")
2732024-08-28T03:59:59.881ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:371
2742024-08-28T03:59:59.928ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
2752024-08-28T03:59:59.935ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_57750eb9-c738-4eb8-89f2-c5eab164d34c/crypt/zone file = sled-storage/src/dataset.rs:414
2762024-08-28T04:00:00.097ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_57750eb9-c738-4eb8-89f2-c5eab164d34c file = sled-storage/src/dataset.rs:499
2772024-08-28T04:00:00.105ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:440
2782024-08-28T04:00:00.105ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:357
2792024-08-28T04:00:00.105ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:342
2802024-08-28T04:00:00.105ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
2812024-08-28T04:00:00.105ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } path = /scratch/u2_1.vdev
2822024-08-28T04:00:00.115ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_1.vdev file = sled-hardware/src/disk.rs:360
2832024-08-28T04:00:00.115ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = 0f906b25-a03c-4334-807e-07a343a51361 (zpool)
2842024-08-28T04:00:00.226ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:307 zpool = ZpoolName { id: 0f906b25-a03c-4334-807e-07a343a51361 (zpool), kind: External }
2852024-08-28T04:00:00.233ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:351
2862024-08-28T04:00:00.233ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:353
2872024-08-28T04:00:00.233ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:357
2882024-08-28T04:00:00.233ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:359
2892024-08-28T04:00:00.233ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key")
2902024-08-28T04:00:00.233ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key")
2912024-08-28T04:00:00.233ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:371
2922024-08-28T04:00:00.237ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/f62e5a13-51f3-4379-acf3-2f8e8601ac1f): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/probe_manager.rs:230 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f
2932024-08-28T04:00:00.280ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
2942024-08-28T04:00:00.288ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_0f906b25-a03c-4334-807e-07a343a51361/crypt/zone file = sled-storage/src/dataset.rs:414
2952024-08-28T04:00:00.439ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_0f906b25-a03c-4334-807e-07a343a51361 file = sled-storage/src/dataset.rs:499
2962024-08-28T04:00:00.447ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:440
2972024-08-28T04:00:00.447ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:357
2982024-08-28T04:00:00.447ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:342
2992024-08-28T04:00:00.447ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3002024-08-28T04:00:00.447ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } path = /scratch/u2_2.vdev
3012024-08-28T04:00:00.457ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_2.vdev file = sled-hardware/src/disk.rs:360
3022024-08-28T04:00:00.457ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = d6170dc9-a1d4-44a9-ba24-00c07a403b1e (zpool)
3032024-08-28T04:00:00.576ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/dataset.rs:307 zpool = ZpoolName { id: d6170dc9-a1d4-44a9-ba24-00c07a403b1e (zpool), kind: External }
3042024-08-28T04:00:00.583ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/dataset.rs:351
3052024-08-28T04:00:00.583ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:353
3062024-08-28T04:00:00.583ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:357
3072024-08-28T04:00:00.583ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:359
3082024-08-28T04:00:00.583ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key")
3092024-08-28T04:00:00.583ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key")
3102024-08-28T04:00:00.583ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:371
3112024-08-28T04:00:00.626ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3122024-08-28T04:00:00.634ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_d6170dc9-a1d4-44a9-ba24-00c07a403b1e/crypt/zone file = sled-storage/src/dataset.rs:414
3132024-08-28T04:00:00.783ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_d6170dc9-a1d4-44a9-ba24-00c07a403b1e file = sled-storage/src/dataset.rs:499
3142024-08-28T04:00:00.790ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:440
3152024-08-28T04:00:00.790ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:357
3162024-08-28T04:00:00.790ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:342
3172024-08-28T04:00:00.790ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3182024-08-28T04:00:00.791ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } path = /scratch/u2_3.vdev
3192024-08-28T04:00:00.800ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_3.vdev file = sled-hardware/src/disk.rs:360
3202024-08-28T04:00:00.800ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = 8955d8a9-1b85-47ea-bd5c-275ef6dc0b1a (zpool)
3212024-08-28T04:00:00.917ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/dataset.rs:307 zpool = ZpoolName { id: 8955d8a9-1b85-47ea-bd5c-275ef6dc0b1a (zpool), kind: External }
3222024-08-28T04:00:00.925ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/dataset.rs:351
3232024-08-28T04:00:00.926ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:353
3242024-08-28T04:00:00.926ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:357
3252024-08-28T04:00:00.926ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:359
3262024-08-28T04:00:00.926ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key")
3272024-08-28T04:00:00.926ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key")
3282024-08-28T04:00:00.926ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:371
3292024-08-28T04:00:00.968ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3302024-08-28T04:00:00.976ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_8955d8a9-1b85-47ea-bd5c-275ef6dc0b1a/crypt/zone file = sled-storage/src/dataset.rs:414
3312024-08-28T04:00:01.129ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_8955d8a9-1b85-47ea-bd5c-275ef6dc0b1a file = sled-storage/src/dataset.rs:499
3322024-08-28T04:00:01.136ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:440
3332024-08-28T04:00:01.136ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:357
3342024-08-28T04:00:01.136ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:342
3352024-08-28T04:00:01.136ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3362024-08-28T04:00:01.136ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } path = /scratch/u2_4.vdev
3372024-08-28T04:00:01.146ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_4.vdev file = sled-hardware/src/disk.rs:360
3382024-08-28T04:00:01.146ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = fb7c10cb-4fd0-49dd-8c21-f83feafbd7dc (zpool)
3392024-08-28T04:00:01.262ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/dataset.rs:307 zpool = ZpoolName { id: fb7c10cb-4fd0-49dd-8c21-f83feafbd7dc (zpool), kind: External }
3402024-08-28T04:00:01.271ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/dataset.rs:351
3412024-08-28T04:00:01.271ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:353
3422024-08-28T04:00:01.271ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:357
3432024-08-28T04:00:01.271ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:359
3442024-08-28T04:00:01.271ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key")
3452024-08-28T04:00:01.271ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key")
3462024-08-28T04:00:01.271ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:371
3472024-08-28T04:00:01.275ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/f62e5a13-51f3-4379-acf3-2f8e8601ac1f): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/probe_manager.rs:230 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f
3482024-08-28T04:00:01.316ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3492024-08-28T04:00:01.323ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_fb7c10cb-4fd0-49dd-8c21-f83feafbd7dc/crypt/zone file = sled-storage/src/dataset.rs:414
3502024-08-28T04:00:01.474ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_fb7c10cb-4fd0-49dd-8c21-f83feafbd7dc file = sled-storage/src/dataset.rs:499
3512024-08-28T04:00:01.482ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:440
3522024-08-28T04:00:01.482ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:357
3532024-08-28T04:00:01.482ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:342
3542024-08-28T04:00:01.482ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3552024-08-28T04:00:01.482ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } path = /scratch/u2_5.vdev
3562024-08-28T04:00:01.492ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_5.vdev file = sled-hardware/src/disk.rs:360
3572024-08-28T04:00:01.492ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = ba3c519e-f98a-4ee2-a345-a21845f5f7a4 (zpool)
3582024-08-28T04:00:01.612ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/dataset.rs:307 zpool = ZpoolName { id: ba3c519e-f98a-4ee2-a345-a21845f5f7a4 (zpool), kind: External }
3592024-08-28T04:00:01.620ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/dataset.rs:351
3602024-08-28T04:00:01.620ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:353
3612024-08-28T04:00:01.620ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:357
3622024-08-28T04:00:01.620ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:359
3632024-08-28T04:00:01.620ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key")
3642024-08-28T04:00:01.620ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key")
3652024-08-28T04:00:01.620ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:371
3662024-08-28T04:00:01.663ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3672024-08-28T04:00:01.671ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_ba3c519e-f98a-4ee2-a345-a21845f5f7a4/crypt/zone file = sled-storage/src/dataset.rs:414
3682024-08-28T04:00:01.827ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_ba3c519e-f98a-4ee2-a345-a21845f5f7a4 file = sled-storage/src/dataset.rs:499
3692024-08-28T04:00:01.836ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:440
3702024-08-28T04:00:01.836ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:357
3712024-08-28T04:00:01.836ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:342
3722024-08-28T04:00:01.836ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3732024-08-28T04:00:01.836ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } path = /scratch/u2_6.vdev
3742024-08-28T04:00:01.846ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_6.vdev file = sled-hardware/src/disk.rs:360
3752024-08-28T04:00:01.847ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = 0913dfa8-99f2-4518-8d37-827ea32c30ba (zpool)
3762024-08-28T04:00:01.962ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/dataset.rs:307 zpool = ZpoolName { id: 0913dfa8-99f2-4518-8d37-827ea32c30ba (zpool), kind: External }
3772024-08-28T04:00:01.970ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/dataset.rs:351
3782024-08-28T04:00:01.970ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:353
3792024-08-28T04:00:01.970ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:357
3802024-08-28T04:00:01.970ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:359
3812024-08-28T04:00:01.970ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key")
3822024-08-28T04:00:01.970ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key")
3832024-08-28T04:00:01.970ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:371
3842024-08-28T04:00:02.018ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3852024-08-28T04:00:02.027ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_0913dfa8-99f2-4518-8d37-827ea32c30ba/crypt/zone file = sled-storage/src/dataset.rs:414
3862024-08-28T04:00:02.181ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_0913dfa8-99f2-4518-8d37-827ea32c30ba file = sled-storage/src/dataset.rs:499
3872024-08-28T04:00:02.189ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:440
3882024-08-28T04:00:02.189ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:357
3892024-08-28T04:00:02.189ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:342
3902024-08-28T04:00:02.189ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3912024-08-28T04:00:02.189ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } path = /scratch/u2_7.vdev
3922024-08-28T04:00:02.199ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_7.vdev file = sled-hardware/src/disk.rs:360
3932024-08-28T04:00:02.199ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = ed7027bd-3cf9-4764-bc15-404c842ab02a (zpool)
3942024-08-28T04:00:02.313ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/dataset.rs:307 zpool = ZpoolName { id: ed7027bd-3cf9-4764-bc15-404c842ab02a (zpool), kind: External }
3952024-08-28T04:00:02.322ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/dataset.rs:351
3962024-08-28T04:00:02.322ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:353
3972024-08-28T04:00:02.322ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:357
3982024-08-28T04:00:02.322ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:359
3992024-08-28T04:00:02.322ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key")
4002024-08-28T04:00:02.322ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key")
4012024-08-28T04:00:02.322ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:371
4022024-08-28T04:00:02.326ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/f62e5a13-51f3-4379-acf3-2f8e8601ac1f): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/probe_manager.rs:230 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f
4032024-08-28T04:00:02.368ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
4042024-08-28T04:00:02.376ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_ed7027bd-3cf9-4764-bc15-404c842ab02a/crypt/zone file = sled-storage/src/dataset.rs:414
4052024-08-28T04:00:02.527ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_ed7027bd-3cf9-4764-bc15-404c842ab02a file = sled-storage/src/dataset.rs:499
4062024-08-28T04:00:02.535ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:440
4072024-08-28T04:00:02.535ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:357
4082024-08-28T04:00:02.535ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:342
4092024-08-28T04:00:02.535ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
4102024-08-28T04:00:02.535ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } path = /scratch/u2_8.vdev
4112024-08-28T04:00:02.544ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_8.vdev file = sled-hardware/src/disk.rs:360
4122024-08-28T04:00:02.544ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = e3fa46c7-4f3a-45f7-afd8-b7799f191587 (zpool)
4132024-08-28T04:00:02.659ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/dataset.rs:307 zpool = ZpoolName { id: e3fa46c7-4f3a-45f7-afd8-b7799f191587 (zpool), kind: External }
4142024-08-28T04:00:02.666ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/dataset.rs:351
4152024-08-28T04:00:02.666ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:353
4162024-08-28T04:00:02.666ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:357
4172024-08-28T04:00:02.667ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:359
4182024-08-28T04:00:02.667ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key")
4192024-08-28T04:00:02.667ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key")
4202024-08-28T04:00:02.667ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:371
4212024-08-28T04:00:02.710ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
4222024-08-28T04:00:02.718ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_e3fa46c7-4f3a-45f7-afd8-b7799f191587/crypt/zone file = sled-storage/src/dataset.rs:414
4232024-08-28T04:00:02.870ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_e3fa46c7-4f3a-45f7-afd8-b7799f191587 file = sled-storage/src/dataset.rs:499
4242024-08-28T04:00:02.880ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:440
4252024-08-28T04:00:02.880ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:357
4262024-08-28T04:00:02.880ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/51625293-e5a8-4cc4-9640-b64176e7a48e/config/.omicron-physical-disks.json.tmp file = common/src/ledger.rs:199 request = omicron_physical_disks_ensure
4272024-08-28T04:00:02.880ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 51625293-e5a8-4cc4-9640-b64176e7a48e (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 4dad4c76-eb6c-4281-9f1d-a63fdb0117e1 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 0f906b25-a03c-4334-807e-07a343a51361 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: d6170dc9-a1d4-44a9-ba24-00c07a403b1e (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 8955d8a9-1b85-47ea-bd5c-275ef6dc0b1a (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: fb7c10cb-4fd0-49dd-8c21-f83feafbd7dc (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: ba3c519e-f98a-4ee2-a345-a21845f5f7a4 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 0913dfa8-99f2-4518-8d37-827ea32c30ba (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: ed7027bd-3cf9-4764-bc15-404c842ab02a (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: e3fa46c7-4f3a-45f7-afd8-b7799f191587 (zpool), kind: External } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
4282024-08-28T04:00:02.880ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
4292024-08-28T04:00:02.880ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
4302024-08-28T04:00:02.880ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/4dad4c76-eb6c-4281-9f1d-a63fdb0117e1/config/.omicron-physical-disks.json.tmp file = common/src/ledger.rs:199 request = omicron_physical_disks_ensure
4312024-08-28T04:00:02.881ZINFOSledAgent: physical disks ensure: Updated storage file = sled-agent/src/sled_agent.rs:834 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f
4322024-08-28T04:00:02.881ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
4332024-08-28T04:00:02.881ZINFOSledAgent: physical disks ensure: Propagating new generation of disks file = sled-agent/src/sled_agent.rs:860 generation = Generation(1) sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f
4342024-08-28T04:00:02.881ZINFOSledAgent: physical disks ensure: Updated storage monitor file = sled-agent/src/sled_agent.rs:865 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f
4352024-08-28T04:00:02.881ZINFOSledAgent: physical disks ensure: Updated zone bundler file = sled-agent/src/sled_agent.rs:870 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f
4362024-08-28T04:00:02.881ZINFOSledAgent (ProbeManager): use_only_these_disks: Processing new request file = sled-agent/src/probe_manager.rs:121 gen = Generation(1) sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f
4372024-08-28T04:00:02.881ZINFOSledAgent: physical disks ensure: Updated probes file = sled-agent/src/sled_agent.rs:876 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f
4382024-08-28T04:00:02.881ZINFOSledAgent (InstanceManager): use_only_these_disks: Processing new request file = sled-agent/src/instance_manager.rs:803 gen = Generation(1)
4392024-08-28T04:00:02.881ZINFOSledAgent: physical disks ensure: Updated instances file = sled-agent/src/sled_agent.rs:881 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f
4402024-08-28T04:00:02.881ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3132366 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:65366 req_id = d0717eae-8c08-4318-8250-25a4fc1823d1 response_code = 200 uri = /omicron-physical-disks
4412024-08-28T04:00:02.913ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(2), zones: [OmicronZoneConfig { id: ae14950e-2bd4-4553-99b0-6488a03a8f3f, underlay_address: fd00:1122:3344:1::1, filesystem_pool: Some(ZpoolName { id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 4e2ba65e-422a-4b1e-b0e3-85f8149a979a, underlay_address: fd00:1122:3344:2::1, filesystem_pool: Some(ZpoolName { id: 0f906b25-a03c-4334-807e-07a343a51361 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0f906b25-a03c-4334-807e-07a343a51361 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 037b07ce-9e39-4982-8c20-c4c38f448b97, underlay_address: fd00:1122:3344:3::1, filesystem_pool: Some(ZpoolName { id: d6170dc9-a1d4-44a9-ba24-00c07a403b1e (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d6170dc9-a1d4-44a9-ba24-00c07a403b1e (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }] } file = sled-agent/src/rack_setup/service.rs:438 sled_address = [fd00:1122:3344:101::1]:12345
4422024-08-28T04:00:02.913ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:60046
4432024-08-28T04:00:02.914ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ae14950e-2bd4-4553-99b0-6488a03a8f3f, dataset_name: DatasetName { pool_name: ZpoolName { id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:406
4442024-08-28T04:00:02.914ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: ae14950e-2bd4-4553-99b0-6488a03a8f3f, dataset_name: DatasetName { pool_name: ZpoolName { id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool), kind: External }, kind: InternalDns }, responder: ... }
4452024-08-28T04:00:02.979ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 4e2ba65e-422a-4b1e-b0e3-85f8149a979a, dataset_name: DatasetName { pool_name: ZpoolName { id: 0f906b25-a03c-4334-807e-07a343a51361 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:406
4462024-08-28T04:00:02.979ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 4e2ba65e-422a-4b1e-b0e3-85f8149a979a, dataset_name: DatasetName { pool_name: ZpoolName { id: 0f906b25-a03c-4334-807e-07a343a51361 (zpool), kind: External }, kind: InternalDns }, responder: ... }
4472024-08-28T04:00:03.045ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 037b07ce-9e39-4982-8c20-c4c38f448b97, dataset_name: DatasetName { pool_name: ZpoolName { id: d6170dc9-a1d4-44a9-ba24-00c07a403b1e (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:406
4482024-08-28T04:00:03.045ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 037b07ce-9e39-4982-8c20-c4c38f448b97, dataset_name: DatasetName { pool_name: ZpoolName { id: d6170dc9-a1d4-44a9-ba24-00c07a403b1e (zpool), kind: External }, kind: InternalDns }, responder: ... }
4492024-08-28T04:00:03.109ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
4502024-08-28T04:00:03.109ZINFOSledAgent (ServiceManager): No ledger in /pool/int/51625293-e5a8-4cc4-9640-b64176e7a48e/config/omicron-zones.json file = common/src/ledger.rs:188
4512024-08-28T04:00:03.109ZINFOSledAgent (ServiceManager): No ledger in /pool/int/4dad4c76-eb6c-4281-9f1d-a63fdb0117e1/config/omicron-zones.json file = common/src/ledger.rs:188
4522024-08-28T04:00:03.109ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
4532024-08-28T04:00:03.153ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
4542024-08-28T04:00:03.153ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
4552024-08-28T04:00:03.153ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
4562024-08-28T04:00:03.205ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_ae14950e-2bd4-4553-99b0-6488a03a8f3f file = illumos-utils/src/zone.rs:316
4572024-08-28T04:00:03.208ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_037b07ce-9e39-4982-8c20-c4c38f448b97 file = illumos-utils/src/zone.rs:316
4582024-08-28T04:00:03.212ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_4e2ba65e-422a-4b1e-b0e3-85f8149a979a file = illumos-utils/src/zone.rs:316
4592024-08-28T04:00:03.224ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_ae14950e-2bd4-4553-99b0-6488a03a8f3f file = illumos-utils/src/zone.rs:354
4602024-08-28T04:00:03.228ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_037b07ce-9e39-4982-8c20-c4c38f448b97 file = illumos-utils/src/zone.rs:354
4612024-08-28T04:00:03.232ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_4e2ba65e-422a-4b1e-b0e3-85f8149a979a file = illumos-utils/src/zone.rs:354
4622024-08-28T04:00:03.329ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/f62e5a13-51f3-4379-acf3-2f8e8601ac1f): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/probe_manager.rs:230 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f
4632024-08-28T04:00:03.759ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 20.623738062s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f
4642024-08-28T04:00:04.697ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_ae14950e-2bd4-4553-99b0-6488a03a8f3f: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService1'/> <propval type="astring" name="gateway" value='fd00:1122:3344:1::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:1::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:1::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:1::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
4652024-08-28T04:00:04.697ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:1::, width: 64 }
4662024-08-28T04:00:04.697ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_ae14950e-2bd4-4553-99b0-6488a03a8f3f zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_ae14950e-2bd4-4553-99b0-6488a03a8f3f
4672024-08-28T04:00:05.181ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_037b07ce-9e39-4982-8c20-c4c38f448b97: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService2'/> <propval type="astring" name="gateway" value='fd00:1122:3344:3::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:3::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:3::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:3::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
4682024-08-28T04:00:05.658ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:3::, width: 64 }
4692024-08-28T04:00:05.658ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_4e2ba65e-422a-4b1e-b0e3-85f8149a979a: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService3'/> <propval type="astring" name="gateway" value='fd00:1122:3344:2::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:2::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:2::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:2::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
4702024-08-28T04:00:05.659ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:2::, width: 64 }
4712024-08-28T04:00:05.666ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_ae14950e-2bd4-4553-99b0-6488a03a8f3f") failed: Property not found. retry in 61.361417ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_ae14950e-2bd4-4553-99b0-6488a03a8f3f
4722024-08-28T04:00:05.666ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_037b07ce-9e39-4982-8c20-c4c38f448b97 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_037b07ce-9e39-4982-8c20-c4c38f448b97
4732024-08-28T04:00:05.669ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_4e2ba65e-422a-4b1e-b0e3-85f8149a979a zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_4e2ba65e-422a-4b1e-b0e3-85f8149a979a
4742024-08-28T04:00:05.740ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_ae14950e-2bd4-4553-99b0-6488a03a8f3f") failed: Property not found. retry in 128.55617ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_ae14950e-2bd4-4553-99b0-6488a03a8f3f
4752024-08-28T04:00:05.793ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_4e2ba65e-422a-4b1e-b0e3-85f8149a979a") failed: Property not found. retry in 39.801391ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_4e2ba65e-422a-4b1e-b0e3-85f8149a979a
4762024-08-28T04:00:05.799ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_037b07ce-9e39-4982-8c20-c4c38f448b97") failed: Property not found. retry in 52.280425ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_037b07ce-9e39-4982-8c20-c4c38f448b97
4772024-08-28T04:00:05.842ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_4e2ba65e-422a-4b1e-b0e3-85f8149a979a") failed: Property not found. retry in 88.698973ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_4e2ba65e-422a-4b1e-b0e3-85f8149a979a
4782024-08-28T04:00:05.893ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_037b07ce-9e39-4982-8c20-c4c38f448b97") failed: Property not found. retry in 140.511103ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_037b07ce-9e39-4982-8c20-c4c38f448b97
4792024-08-28T04:00:05.955ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_ae14950e-2bd4-4553-99b0-6488a03a8f3f") failed: Property not found. retry in 133.096292ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_ae14950e-2bd4-4553-99b0-6488a03a8f3f
4802024-08-28T04:00:05.974ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_4e2ba65e-422a-4b1e-b0e3-85f8149a979a") failed: Property not found. retry in 227.597919ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_4e2ba65e-422a-4b1e-b0e3-85f8149a979a
4812024-08-28T04:00:06.127ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_037b07ce-9e39-4982-8c20-c4c38f448b97") failed: Property not found. retry in 247.873848ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_037b07ce-9e39-4982-8c20-c4c38f448b97
4822024-08-28T04:00:06.144ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_ae14950e-2bd4-4553-99b0-6488a03a8f3f") failed: Property not found. retry in 581.095876ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_ae14950e-2bd4-4553-99b0-6488a03a8f3f
4832024-08-28T04:00:06.223ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_4e2ba65e-422a-4b1e-b0e3-85f8149a979a") failed: Property not found. retry in 464.357372ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_4e2ba65e-422a-4b1e-b0e3-85f8149a979a
4842024-08-28T04:00:06.386ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_037b07ce-9e39-4982-8c20-c4c38f448b97") failed: Property not found. retry in 471.452061ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_037b07ce-9e39-4982-8c20-c4c38f448b97
4852024-08-28T04:00:06.695ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_4e2ba65e-422a-4b1e-b0e3-85f8149a979a") failed: Property not found. retry in 652.027934ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_4e2ba65e-422a-4b1e-b0e3-85f8149a979a
4862024-08-28T04:00:06.733ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_ae14950e-2bd4-4553-99b0-6488a03a8f3f") failed: Property not found. retry in 428.402622ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_ae14950e-2bd4-4553-99b0-6488a03a8f3f
4872024-08-28T04:00:06.867ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_037b07ce-9e39-4982-8c20-c4c38f448b97") failed: Property not found. retry in 879.512151ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_037b07ce-9e39-4982-8c20-c4c38f448b97
4882024-08-28T04:00:07.172ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_ae14950e-2bd4-4553-99b0-6488a03a8f3f") failed: Property not found. retry in 551.351451ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_ae14950e-2bd4-4553-99b0-6488a03a8f3f
4892024-08-28T04:00:07.358ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_4e2ba65e-422a-4b1e-b0e3-85f8149a979a") failed: Property not found. retry in 818.245165ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_4e2ba65e-422a-4b1e-b0e3-85f8149a979a
4902024-08-28T04:00:07.732ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_ae14950e-2bd4-4553-99b0-6488a03a8f3f") failed: Property not found. retry in 786.373848ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_ae14950e-2bd4-4553-99b0-6488a03a8f3f
4912024-08-28T04:00:07.756ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_037b07ce-9e39-4982-8c20-c4c38f448b97") failed: Property not found. retry in 1.258124749s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_037b07ce-9e39-4982-8c20-c4c38f448b97
4922024-08-28T04:00:08.186ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_4e2ba65e-422a-4b1e-b0e3-85f8149a979a") failed: Property not found. retry in 1.22502882s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_4e2ba65e-422a-4b1e-b0e3-85f8149a979a
4932024-08-28T04:00:08.351ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::2 exists - OK file = sled-agent/src/services.rs:4206
4942024-08-28T04:00:08.351ZINFOSledAgent (ServiceManager): Ensuring there is a default route file = sled-agent/src/services.rs:4216 gateway = fd00:1122:3344:101::1
4952024-08-28T04:00:08.357ZINFOSledAgent (ServiceManager): configuring dendrite service file = sled-agent/src/services.rs:4295
4962024-08-28T04:00:08.481ZINFOSledAgent (ServiceManager): refreshed dendrite service with new configuration file = sled-agent/src/services.rs:4343
4972024-08-28T04:00:08.481ZINFOSledAgent (ServiceManager): configuring lldp service file = sled-agent/src/services.rs:4370
4982024-08-28T04:00:08.520ZINFOSledAgent (ServiceManager): refreshed lldpd service with new configuration file = sled-agent/src/services.rs:4383
4992024-08-28T04:00:08.520ZINFOSledAgent (ServiceManager): configuring MGS service file = sled-agent/src/services.rs:4247
5002024-08-28T04:00:08.536ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_ae14950e-2bd4-4553-99b0-6488a03a8f3f") failed: Property not found. retry in 830.36254ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_ae14950e-2bd4-4553-99b0-6488a03a8f3f
5012024-08-28T04:00:08.580ZINFOSledAgent (ServiceManager): refreshed MGS service with new configuration file = sled-agent/src/services.rs:4289
5022024-08-28T04:00:08.580ZINFOSledAgent (ServiceManager): configuring wicketd file = sled-agent/src/services.rs:4350 rack_subnet = fd00:1122:3344::
5032024-08-28T04:00:08.600ZINFOSledAgent (ServiceManager): refreshed wicketd service with new configuration file = sled-agent/src/services.rs:4361
5042024-08-28T04:00:08.600ZINFOSledAgent (ServiceManager): configuring mgd service file = sled-agent/src/services.rs:4403
5052024-08-28T04:00:08.692ZINFOSledAgent (ServiceManager): refreshed mgd service with new configuration file = sled-agent/src/services.rs:4435
5062024-08-28T04:00:08.692ZINFOSledAgent (ServiceManager): configuring mg-ddm service file = sled-agent/src/services.rs:4441
5072024-08-28T04:00:08.801ZINFOSledAgent (ServiceManager): refreshed mg-ddm service with new configuration file = sled-agent/src/services.rs:4482
5082024-08-28T04:00:08.801ZINFOSledAgent (ServiceManager): Determining physical location of our switch zone at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:366
5092024-08-28T04:00:08.834ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/f62e5a13-51f3-4379-acf3-2f8e8601ac1f): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/nexus.rs:571 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f
5102024-08-28T04:00:08.834ZWARNSledAgent (ServiceManager): Failed to get switch ID from MGS (retrying in 35.518238ms) error = Communication Error: error sending request for url (http://[fd00:1122:3344:101::2]:12225/local/switch-id): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/bootstrap/early_networking.rs:385
5112024-08-28T04:00:08.871ZWARNSledAgent (ServiceManager): Failed to get switch ID from MGS (retrying in 106.731875ms) error = Communication Error: error sending request for url (http://[fd00:1122:3344:101::2]:12225/local/switch-id): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/bootstrap/early_networking.rs:385
5122024-08-28T04:00:08.979ZWARNSledAgent (ServiceManager): Failed to get switch ID from MGS (retrying in 186.075366ms) error = Communication Error: error sending request for url (http://[fd00:1122:3344:101::2]:12225/local/switch-id): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/bootstrap/early_networking.rs:385
5132024-08-28T04:00:09.022ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_037b07ce-9e39-4982-8c20-c4c38f448b97") failed: Property not found. retry in 1.104910587s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_037b07ce-9e39-4982-8c20-c4c38f448b97
5142024-08-28T04:00:09.166ZWARNSledAgent (ServiceManager): Failed to get switch ID from MGS (retrying in 332.684305ms) error = Communication Error: error sending request for url (http://[fd00:1122:3344:101::2]:12225/local/switch-id): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/bootstrap/early_networking.rs:385
5152024-08-28T04:00:09.377ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_ae14950e-2bd4-4553-99b0-6488a03a8f3f") failed: Property not found. retry in 582.336379ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_ae14950e-2bd4-4553-99b0-6488a03a8f3f
5162024-08-28T04:00:09.420ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_4e2ba65e-422a-4b1e-b0e3-85f8149a979a") failed: Property not found. retry in 1.138265241s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_4e2ba65e-422a-4b1e-b0e3-85f8149a979a
5172024-08-28T04:00:09.424ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::2]:54658
5182024-08-28T04:00:09.425ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 73 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::2]:54658 req_id = e3298fe4-f116-4bcc-91ad-c55e0f7c99ef response_code = 200 uri = /sled-identifiers
5192024-08-28T04:00:09.499ZINFOSledAgent (ServiceManager): Initializing 1 Uplinks on Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:414
5202024-08-28T04:00:09.532ZINFOSledAgent (ServiceManager): Building Port Configuration file = sled-agent/src/bootstrap/early_networking.rs:676
5212024-08-28T04:00:09.532ZINFOSledAgent (ServiceManager): Checking dendrite uptime file = sled-agent/src/bootstrap/early_networking.rs:714
5222024-08-28T04:00:09.532ZINFOSledAgent (ServiceManager): Dendrite online file = sled-agent/src/bootstrap/early_networking.rs:717 uptime = 0
5232024-08-28T04:00:09.532ZINFOSledAgent (ServiceManager): Configuring default uplink on switch config = PortSettings {\n links: {\n "0": LinkSettings {\n addrs: [\n 10.151.2.174,\n ],\n params: LinkCreate {\n autoneg: false,\n fec: None,\n kr: false,\n lane: Some(\n LinkId(\n 0,\n ),\n ),\n speed: Speed40G,\n },\n },\n },\n} file = sled-agent/src/bootstrap/early_networking.rs:436
5242024-08-28T04:00:09.569ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
5252024-08-28T04:00:09.618ZINFOSledAgent (StorageManager): Received DetectedRawDisksChanged { raw_disks: {}, tx: ... } file = sled-storage/src/manager.rs:406
5262024-08-28T04:00:09.618ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
5272024-08-28T04:00:09.618ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
5282024-08-28T04:00:09.618ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
5292024-08-28T04:00:09.618ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
5302024-08-28T04:00:09.618ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
5312024-08-28T04:00:09.618ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
5322024-08-28T04:00:09.618ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
5332024-08-28T04:00:09.618ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
5342024-08-28T04:00:09.618ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
5352024-08-28T04:00:09.618ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
5362024-08-28T04:00:09.618ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
5372024-08-28T04:00:09.618ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
5382024-08-28T04:00:09.618ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
5392024-08-28T04:00:09.618ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
5402024-08-28T04:00:09.618ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
5412024-08-28T04:00:09.618ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
5422024-08-28T04:00:09.618ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
5432024-08-28T04:00:09.618ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
5442024-08-28T04:00:09.618ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
5452024-08-28T04:00:09.618ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
5462024-08-28T04:00:09.618ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
5472024-08-28T04:00:09.618ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
5482024-08-28T04:00:09.969ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_ae14950e-2bd4-4553-99b0-6488a03a8f3f") failed: Property not found. retry in 1.33424143s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_ae14950e-2bd4-4553-99b0-6488a03a8f3f
5492024-08-28T04:00:10.135ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_037b07ce-9e39-4982-8c20-c4c38f448b97") failed: Property not found. retry in 1.347662425s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_037b07ce-9e39-4982-8c20-c4c38f448b97
5502024-08-28T04:00:10.568ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_4e2ba65e-422a-4b1e-b0e3-85f8149a979a") failed: Property not found. retry in 898.291575ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_4e2ba65e-422a-4b1e-b0e3-85f8149a979a
5512024-08-28T04:00:11.501ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_internal_dns_ae14950e-2bd4-4553-99b0-6488a03a8f3f
5522024-08-28T04:00:11.501ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_internal_dns_037b07ce-9e39-4982-8c20-c4c38f448b97
5532024-08-28T04:00:11.501ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_internal_dns_4e2ba65e-422a-4b1e-b0e3-85f8149a979a
5542024-08-28T04:00:11.501ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/51625293-e5a8-4cc4-9640-b64176e7a48e/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
5552024-08-28T04:00:11.501ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/4dad4c76-eb6c-4281-9f1d-a63fdb0117e1/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
5562024-08-28T04:00:11.502ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8588357 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:60046 req_id = b904ae7b-d930-4bb6-a0e0-1cfc71744cd1 response_code = 204 uri = /omicron-zones
5572024-08-28T04:00:11.503ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:1::1]:5353 file = sled-agent/src/rack_setup/service.rs:555
5582024-08-28T04:00:11.580ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:2::1]:5353 file = sled-agent/src/rack_setup/service.rs:555
5592024-08-28T04:00:11.644ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:3::1]:5353 file = sled-agent/src/rack_setup/service.rs:555
5602024-08-28T04:00:11.705ZINFOSledAgent (RSS): Configured all DNS servers file = sled-agent/src/rack_setup/service.rs:591
5612024-08-28T04:00:11.705ZINFOSledAgent (RSS): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
5622024-08-28T04:00:11.739ZINFOSledAgent (RSS): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
5632024-08-28T04:00:11.740ZINFOSledAgent (RSS): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
5642024-08-28T04:00:11.740ZINFOSledAgent (RSS): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
5652024-08-28T04:00:11.770ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(3), zones: [OmicronZoneConfig { id: ae14950e-2bd4-4553-99b0-6488a03a8f3f, underlay_address: fd00:1122:3344:1::1, filesystem_pool: Some(ZpoolName { id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 4e2ba65e-422a-4b1e-b0e3-85f8149a979a, underlay_address: fd00:1122:3344:2::1, filesystem_pool: Some(ZpoolName { id: 0f906b25-a03c-4334-807e-07a343a51361 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0f906b25-a03c-4334-807e-07a343a51361 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 037b07ce-9e39-4982-8c20-c4c38f448b97, underlay_address: fd00:1122:3344:3::1, filesystem_pool: Some(ZpoolName { id: d6170dc9-a1d4-44a9-ba24-00c07a403b1e (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d6170dc9-a1d4-44a9-ba24-00c07a403b1e (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 277864c2-af7f-46e4-a527-d1f7bc0d990d, underlay_address: fd00:1122:3344:101::1b, filesystem_pool: Some(ZpoolName { id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 870956f8-434c-40b0-9f42-ba578decd127, kind: Service { id: 277864c2-af7f-46e4-a527-d1f7bc0d990d }, name: Name("ntp-277864c2-af7f-46e4-a527-d1f7bc0d990d"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 181, 11])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }] } file = sled-agent/src/rack_setup/service.rs:438 sled_address = [fd00:1122:3344:101::1]:12345
5662024-08-28T04:00:11.770ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:39211
5672024-08-28T04:00:11.770ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ae14950e-2bd4-4553-99b0-6488a03a8f3f, dataset_name: DatasetName { pool_name: ZpoolName { id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:406
5682024-08-28T04:00:11.770ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: ae14950e-2bd4-4553-99b0-6488a03a8f3f, dataset_name: DatasetName { pool_name: ZpoolName { id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool), kind: External }, kind: InternalDns }, responder: ... }
5692024-08-28T04:00:11.791ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 4e2ba65e-422a-4b1e-b0e3-85f8149a979a, dataset_name: DatasetName { pool_name: ZpoolName { id: 0f906b25-a03c-4334-807e-07a343a51361 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:406
5702024-08-28T04:00:11.791ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 4e2ba65e-422a-4b1e-b0e3-85f8149a979a, dataset_name: DatasetName { pool_name: ZpoolName { id: 0f906b25-a03c-4334-807e-07a343a51361 (zpool), kind: External }, kind: InternalDns }, responder: ... }
5712024-08-28T04:00:11.811ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 037b07ce-9e39-4982-8c20-c4c38f448b97, dataset_name: DatasetName { pool_name: ZpoolName { id: d6170dc9-a1d4-44a9-ba24-00c07a403b1e (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:406
5722024-08-28T04:00:11.811ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 037b07ce-9e39-4982-8c20-c4c38f448b97, dataset_name: DatasetName { pool_name: ZpoolName { id: d6170dc9-a1d4-44a9-ba24-00c07a403b1e (zpool), kind: External }, kind: InternalDns }, responder: ... }
5732024-08-28T04:00:11.826ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
5742024-08-28T04:00:11.827ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/51625293-e5a8-4cc4-9640-b64176e7a48e/config/omicron-zones.json file = common/src/ledger.rs:177
5752024-08-28T04:00:11.827ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/4dad4c76-eb6c-4281-9f1d-a63fdb0117e1/config/omicron-zones.json file = common/src/ledger.rs:177
5762024-08-28T04:00:11.827ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
5772024-08-28T04:00:11.837ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
5782024-08-28T04:00:11.870ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
5792024-08-28T04:00:11.871ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
5802024-08-28T04:00:11.871ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
5812024-08-28T04:00:11.902ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte0", ip: 172.30.3.5, mac: MacAddr6([168, 64, 37, 255, 181, 11]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.3.1 } }) }
5822024-08-28T04:00:11.902ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1235 zone_type = boundary_ntp
5832024-08-28T04:00:11.903ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.169 file = clients/dpd-client/src/lib.rs:83
5842024-08-28T04:00:11.903ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.169 file = clients/dpd-client/src/lib.rs:90
5852024-08-28T04:00:11.904ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.169 file = clients/dpd-client/src/lib.rs:118
5862024-08-28T04:00:11.904ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
5872024-08-28T04:00:11.928ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_ntp_277864c2-af7f-46e4-a527-d1f7bc0d990d file = illumos-utils/src/zone.rs:316
5882024-08-28T04:00:11.945ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_ntp_277864c2-af7f-46e4-a527-d1f7bc0d990d file = illumos-utils/src/zone.rs:354
5892024-08-28T04:00:12.574ZINFOSledAgent (ServiceManager): Profile for oxz_ntp_277864c2-af7f-46e4-a527-d1f7bc0d990d: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService4'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::1b'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/chrony-setup"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="allow" value='fd00:1122:3344:100::/56'/> <propval type="boolean" name="boundary" value='true'/> <propval type="astring" name="boundary_pool" value='boundary-ntp.control-plane.oxide.internal'/> <propval type="astring" name="server" value='0.pool.ntp.org'/> </property_group> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="1.1.1.1" /> <value_node value="9.9.9.9" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/ntp"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte0'/> <propval type="astring" name="gateway" value='172.30.3.1'/> <propval type="astring" name="ip" value='172.30.3.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
5902024-08-28T04:00:12.574ZINFOSledAgent (ServiceManager): Booting oxz_ntp_277864c2-af7f-46e4-a527-d1f7bc0d990d zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_ntp_277864c2-af7f-46e4-a527-d1f7bc0d990d
5912024-08-28T04:00:12.697ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_277864c2-af7f-46e4-a527-d1f7bc0d990d") failed: Property not found. retry in 63.637078ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_277864c2-af7f-46e4-a527-d1f7bc0d990d
5922024-08-28T04:00:12.786ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_277864c2-af7f-46e4-a527-d1f7bc0d990d") failed: Property not found. retry in 85.359375ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_277864c2-af7f-46e4-a527-d1f7bc0d990d
5932024-08-28T04:00:12.899ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_277864c2-af7f-46e4-a527-d1f7bc0d990d") failed: Property not found. retry in 104.390896ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_277864c2-af7f-46e4-a527-d1f7bc0d990d
5942024-08-28T04:00:13.016ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_277864c2-af7f-46e4-a527-d1f7bc0d990d") failed: Property not found. retry in 303.234898ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_277864c2-af7f-46e4-a527-d1f7bc0d990d
5952024-08-28T04:00:13.328ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_277864c2-af7f-46e4-a527-d1f7bc0d990d") failed: Property not found. retry in 1.113330672s file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_277864c2-af7f-46e4-a527-d1f7bc0d990d
5962024-08-28T04:00:14.451ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_277864c2-af7f-46e4-a527-d1f7bc0d990d") failed: Property not found. retry in 1.091260889s file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_277864c2-af7f-46e4-a527-d1f7bc0d990d
5972024-08-28T04:00:15.551ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_277864c2-af7f-46e4-a527-d1f7bc0d990d") failed: Property not found. retry in 613.670946ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_277864c2-af7f-46e4-a527-d1f7bc0d990d
5982024-08-28T04:00:16.173ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_277864c2-af7f-46e4-a527-d1f7bc0d990d") failed: Property not found. retry in 542.720602ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_277864c2-af7f-46e4-a527-d1f7bc0d990d
5992024-08-28T04:00:16.725ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_277864c2-af7f-46e4-a527-d1f7bc0d990d") failed: Property not found. retry in 1.213550214s file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_277864c2-af7f-46e4-a527-d1f7bc0d990d
6002024-08-28T04:00:17.958ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_ntp_277864c2-af7f-46e4-a527-d1f7bc0d990d
6012024-08-28T04:00:17.958ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/51625293-e5a8-4cc4-9640-b64176e7a48e/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
6022024-08-28T04:00:17.959ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/4dad4c76-eb6c-4281-9f1d-a63fdb0117e1/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
6032024-08-28T04:00:17.959ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 6188873 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:39211 req_id = b743ca93-4119-49d7-a274-34d5e1780cd1 response_code = 204 uri = /omicron-zones
6042024-08-28T04:00:17.962ZINFOSledAgent (RSS): Waiting for rack time synchronization file = sled-agent/src/rack_setup/service.rs:632
6052024-08-28T04:00:17.993ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6062024-08-28T04:00:17.993ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:58755
6072024-08-28T04:00:18.003ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_277864c2-af7f-46e4-a527-d1f7bc0d990d': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3847
6082024-08-28T04:00:18.003ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:902 latency_us = 10690 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:58755 req_id = 65a99793-8607-4704-bd9b-8e520cbc38b5 response_code = 500 uri = /timesync
6092024-08-28T04:00:18.006ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6102024-08-28T04:00:18.116ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6112024-08-28T04:00:18.116ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:43846
6122024-08-28T04:00:22.125ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_277864c2-af7f-46e4-a527-d1f7bc0d990d': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3847
6132024-08-28T04:00:22.125ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:902 latency_us = 4009100 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:43846 req_id = c796ac70-c682-44ef-9e0d-68b738a94be9 response_code = 500 uri = /timesync
6142024-08-28T04:00:22.129ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6152024-08-28T04:00:22.264ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6162024-08-28T04:00:22.264ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:47451
6172024-08-28T04:00:22.272ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_277864c2-af7f-46e4-a527-d1f7bc0d990d': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3847
6182024-08-28T04:00:22.273ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:902 latency_us = 8377 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:47451 req_id = 362a7029-22f1-4153-a585-d39f706c034a response_code = 500 uri = /timesync
6192024-08-28T04:00:22.275ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6202024-08-28T04:00:22.464ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6212024-08-28T04:00:22.464ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:58096
6222024-08-28T04:00:22.472ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_277864c2-af7f-46e4-a527-d1f7bc0d990d': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3847
6232024-08-28T04:00:22.472ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:902 latency_us = 8376 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:58096 req_id = 7f9f5b87-fc74-4f71-8811-f1b60967ed01 response_code = 500 uri = /timesync
6242024-08-28T04:00:22.474ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6252024-08-28T04:00:22.662ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6262024-08-28T04:00:22.662ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:64025
6272024-08-28T04:00:24.147ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2009399 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:64025 req_id = 5b9ca048-315a-4fd2-9b24-9e7b9c834118 response_code = 200 uri = /timesync
6282024-08-28T04:00:24.152ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:640
6292024-08-28T04:00:24.152ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6302024-08-28T04:00:24.288ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6312024-08-28T04:00:24.288ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:53053
6322024-08-28T04:00:24.296ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8385 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:53053 req_id = d9a838d6-e750-45e2-90a3-f840d3154097 response_code = 200 uri = /timesync
6332024-08-28T04:00:24.298ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:640
6342024-08-28T04:00:24.298ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6352024-08-28T04:00:25.152ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6362024-08-28T04:00:25.152ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:45914
6372024-08-28T04:00:25.160ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8368 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:45914 req_id = 4ce3931f-26d1-450e-984d-3629f9859bdd response_code = 200 uri = /timesync
6382024-08-28T04:00:25.162ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: -0.502249539 } file = sled-agent/src/rack_setup/service.rs:640
6392024-08-28T04:00:25.162ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6402024-08-28T04:00:25.439ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6412024-08-28T04:00:25.439ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:61942
6422024-08-28T04:00:25.446ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 7757 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:61942 req_id = ba9dad8c-9033-47d6-a4ed-87f713c98f21 response_code = 200 uri = /timesync
6432024-08-28T04:00:25.448ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: -0.501472592 } file = sled-agent/src/rack_setup/service.rs:640
6442024-08-28T04:00:25.448ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6452024-08-28T04:00:25.941ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6462024-08-28T04:00:25.941ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:43619
6472024-08-28T04:00:25.949ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8136 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:43619 req_id = 58df572d-cb73-46ce-afc4-f77825433185 response_code = 200 uri = /timesync
6482024-08-28T04:00:25.952ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: -0.499104589 } file = sled-agent/src/rack_setup/service.rs:640
6492024-08-28T04:00:25.952ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6502024-08-28T04:00:26.387ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6512024-08-28T04:00:26.387ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:60204
6522024-08-28T04:00:26.394ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 7087 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:60204 req_id = 7b2f4db5-83c6-4b9f-aef2-911f0aa4598f response_code = 200 uri = /timesync
6532024-08-28T04:00:26.397ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1724817624.173996, correction: -0.498228222 } file = sled-agent/src/rack_setup/service.rs:640
6542024-08-28T04:00:26.397ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6552024-08-28T04:00:27.062ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6562024-08-28T04:00:27.062ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:57571
6572024-08-28T04:00:27.070ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8060 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:57571 req_id = d4742074-3e3c-4014-b217-06a41f548bdc response_code = 200 uri = /timesync
6582024-08-28T04:00:27.073ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1724817624.173996, correction: -0.496082217 } file = sled-agent/src/rack_setup/service.rs:640
6592024-08-28T04:00:27.073ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6602024-08-28T04:00:27.765ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6612024-08-28T04:00:27.765ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:40723
6622024-08-28T04:00:27.774ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8353 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:40723 req_id = d7b0e0f6-712c-4af9-8a5c-cd21fa5fc196 response_code = 200 uri = /timesync
6632024-08-28T04:00:27.775ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1724817624.173996, correction: -0.494178653 } file = sled-agent/src/rack_setup/service.rs:640
6642024-08-28T04:00:27.775ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6652024-08-28T04:00:28.537ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6662024-08-28T04:00:28.537ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:50146
6672024-08-28T04:00:28.545ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 7898 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:50146 req_id = dc37acd1-babd-43f2-9ec4-d2b56a3ed15e response_code = 200 uri = /timesync
6682024-08-28T04:00:28.547ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 1757546723, ip_addr: 104.194.8.227, stratum: 3, ref_time: 1724817627.7033024, correction: -0.492051125 } file = sled-agent/src/rack_setup/service.rs:640
6692024-08-28T04:00:28.547ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6702024-08-28T04:00:29.230ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6712024-08-28T04:00:29.231ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:44001
6722024-08-28T04:00:29.238ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 7901 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:44001 req_id = 3c328801-d0e1-4611-8861-561a9171d7ae response_code = 200 uri = /timesync
6732024-08-28T04:00:29.241ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1724817624.173996, correction: -0.490168452 } file = sled-agent/src/rack_setup/service.rs:640
6742024-08-28T04:00:29.241ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6752024-08-28T04:00:30.087ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6762024-08-28T04:00:30.088ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:50837
6772024-08-28T04:00:30.096ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8633 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:50837 req_id = d1a55523-fc25-48a7-9e7d-d963a00b0487 response_code = 200 uri = /timesync
6782024-08-28T04:00:30.098ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1724817624.173996, correction: -0.487927645 } file = sled-agent/src/rack_setup/service.rs:640
6792024-08-28T04:00:30.098ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6802024-08-28T04:00:30.931ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6812024-08-28T04:00:30.931ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:37662
6822024-08-28T04:00:30.939ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8266 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:37662 req_id = b0b7eba2-52a9-4d44-b79c-7ccbe50fa4e0 response_code = 200 uri = /timesync
6832024-08-28T04:00:30.942ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 1757546723, ip_addr: 104.194.8.227, stratum: 3, ref_time: 1724817629.2914486, correction: -0.48547408 } file = sled-agent/src/rack_setup/service.rs:640
6842024-08-28T04:00:30.942ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6852024-08-28T04:00:32.381ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6862024-08-28T04:00:32.381ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:46315
6872024-08-28T04:00:32.389ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8278 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:46315 req_id = d5057f30-49f0-4482-b027-841d2a610f2e response_code = 200 uri = /timesync
6882024-08-28T04:00:32.392ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 1757546723, ip_addr: 104.194.8.227, stratum: 3, ref_time: 1724817631.051216, correction: -0.48121202 } file = sled-agent/src/rack_setup/service.rs:640
6892024-08-28T04:00:32.392ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6902024-08-28T04:00:33.646ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6912024-08-28T04:00:33.646ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:55817
6922024-08-28T04:00:33.655ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 9085 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:55817 req_id = 52303182-f2ca-44d7-8687-1552f1aae0b6 response_code = 200 uri = /timesync
6932024-08-28T04:00:33.658ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 1757546723, ip_addr: 104.194.8.227, stratum: 3, ref_time: 1724817632.8660626, correction: -0.477808535 } file = sled-agent/src/rack_setup/service.rs:640
6942024-08-28T04:00:33.658ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6952024-08-28T04:00:35.341ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6962024-08-28T04:00:35.341ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:40064
6972024-08-28T04:00:35.350ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8758 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:40064 req_id = e693fdde-812d-42b3-80d4-9be124853127 response_code = 200 uri = /timesync
6982024-08-28T04:00:35.353ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 1757546723, ip_addr: 104.194.8.227, stratum: 3, ref_time: 1724817633.8720376, correction: -0.47311005 } file = sled-agent/src/rack_setup/service.rs:640
6992024-08-28T04:00:35.353ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
7002024-08-28T04:00:37.664ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
7012024-08-28T04:00:37.664ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:35778
7022024-08-28T04:00:37.672ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8318 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:35778 req_id = e6de2318-6f5a-4780-82ac-66f491cf151f response_code = 200 uri = /timesync
7032024-08-28T04:00:37.675ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 1757546723, ip_addr: 104.194.8.227, stratum: 3, ref_time: 1724817636.6146512, correction: -0.466847062 } file = sled-agent/src/rack_setup/service.rs:640
7042024-08-28T04:00:37.675ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
7052024-08-28T04:00:38.695ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:38744
7062024-08-28T04:00:38.695ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 78 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:38744 req_id = 114e3df5-49e6-4654-b392-59e1982f8948 response_code = 200 uri = /baseboard
7072024-08-28T04:00:39.836ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 137.503326ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f
7082024-08-28T04:00:41.293ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
7092024-08-28T04:00:41.293ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:62951
7102024-08-28T04:00:41.302ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 9099 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:62951 req_id = bd8815fa-1cc1-4efd-8e01-cb2fddfb8e15 response_code = 200 uri = /timesync
7112024-08-28T04:00:41.305ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 1757546723, ip_addr: 104.194.8.227, stratum: 3, ref_time: 1724817639.781906, correction: -0.456971973 } file = sled-agent/src/rack_setup/service.rs:640
7122024-08-28T04:00:41.305ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
7132024-08-28T04:00:44.238ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
7142024-08-28T04:00:44.238ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:64146
7152024-08-28T04:00:44.247ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 9022 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:64146 req_id = 4407cbd0-d2ca-4df6-9823-fe1744862cd9 response_code = 200 uri = /timesync
7162024-08-28T04:00:44.250ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 1757546723, ip_addr: 104.194.8.227, stratum: 3, ref_time: 1724817642.9502633, correction: -0.44893983 } file = sled-agent/src/rack_setup/service.rs:640
7172024-08-28T04:00:44.250ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
7182024-08-28T04:00:50.349ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
7192024-08-28T04:00:50.349ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:58564
7202024-08-28T04:00:50.358ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8883 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:58564 req_id = c379a12a-809c-4618-8b7e-d81fad49df95 response_code = 200 uri = /timesync
7212024-08-28T04:00:50.361ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 1757546723, ip_addr: 104.194.8.227, stratum: 3, ref_time: 1724817647.618745, correction: -0.432343692 } file = sled-agent/src/rack_setup/service.rs:640
7222024-08-28T04:00:50.361ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
7232024-08-28T04:00:54.968ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 506.835577ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f
7242024-08-28T04:00:55.232ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
7252024-08-28T04:00:55.232ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:59355
7262024-08-28T04:00:55.242ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 9759 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:59355 req_id = 81a34f47-840c-43da-bad4-92ef73ab7ba5 response_code = 200 uri = /timesync
7272024-08-28T04:00:55.244ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 1757546723, ip_addr: 104.194.8.227, stratum: 3, ref_time: 1724817654.1063008, correction: -0.419074893 } file = sled-agent/src/rack_setup/service.rs:640
7282024-08-28T04:00:55.244ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
7292024-08-28T04:01:01.584ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
7302024-08-28T04:01:01.584ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:54314
7312024-08-28T04:01:01.594ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 9752 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:54314 req_id = 102ae7e2-f680-47e5-96c0-9d075c2dc67d response_code = 200 uri = /timesync
7322024-08-28T04:01:01.597ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 1757546723, ip_addr: 104.194.8.227, stratum: 3, ref_time: 1724817658.2510843, correction: -0.401812881 } file = sled-agent/src/rack_setup/service.rs:640
7332024-08-28T04:01:01.597ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
7342024-08-28T04:01:05.717ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
7352024-08-28T04:01:05.717ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:62246
7362024-08-28T04:01:05.726ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8877 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:62246 req_id = 8a93211f-c56d-45c3-84ab-9b2236d760a5 response_code = 200 uri = /timesync
7372024-08-28T04:01:05.729ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 1757546723, ip_addr: 104.194.8.227, stratum: 3, ref_time: 1724817661.6202776, correction: -0.390578955 } file = sled-agent/src/rack_setup/service.rs:640
7382024-08-28T04:01:05.729ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
7392024-08-28T04:01:08.646ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:43237
7402024-08-28T04:01:08.646ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 50 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:43237 req_id = 70d93801-bcc3-4b76-8067-ebf7c5157e84 response_code = 200 uri = /baseboard
7412024-08-28T04:01:10.468ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 635.44796ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f
7422024-08-28T04:01:13.419ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
7432024-08-28T04:01:13.419ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:65387
7442024-08-28T04:01:13.428ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 9156 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:65387 req_id = 879aac66-e61f-4653-8e95-7cea09f3afed response_code = 200 uri = /timesync
7452024-08-28T04:01:13.430ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 1757546723, ip_addr: 104.194.8.227, stratum: 3, ref_time: 1724817669.7081025, correction: -0.369772404 } file = sled-agent/src/rack_setup/service.rs:640
7462024-08-28T04:01:13.430ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
7472024-08-28T04:01:24.714ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
7482024-08-28T04:01:24.714ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:43335
7492024-08-28T04:01:24.723ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8777 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:43335 req_id = 279cd35d-d0d8-4d4a-951a-3cf47f30bd51 response_code = 200 uri = /timesync
7502024-08-28T04:01:24.726ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 1757546723, ip_addr: 104.194.8.227, stratum: 3, ref_time: 1724817677.8527856, correction: -0.339088202 } file = sled-agent/src/rack_setup/service.rs:640
7512024-08-28T04:01:24.726ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
7522024-08-28T04:01:26.095ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 2.662126569s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f
7532024-08-28T04:01:38.598ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:53387
7542024-08-28T04:01:38.598ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 49 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:53387 req_id = 8ddfed1e-63a6-40d7-924c-67040a9d5e25 response_code = 200 uri = /baseboard
7552024-08-28T04:01:39.142ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
7562024-08-28T04:01:39.143ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:59176
7572024-08-28T04:01:39.152ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 9079 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:59176 req_id = c727afb4-3f17-4aec-a17c-16ef05ff6ef6 response_code = 200 uri = /timesync
7582024-08-28T04:01:39.155ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 1757546723, ip_addr: 104.194.8.227, stratum: 3, ref_time: 1724817694.6993773, correction: -0.299851507 } file = sled-agent/src/rack_setup/service.rs:640
7592024-08-28T04:01:39.155ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
7602024-08-28T04:01:43.745ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 4.823520549s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f
7612024-08-28T04:01:50.213ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
7622024-08-28T04:01:50.213ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:59678
7632024-08-28T04:01:50.224ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 10908 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:59678 req_id = 7792ffda-f148-4866-8fb9-d97ed9af0c3b response_code = 200 uri = /timesync
7642024-08-28T04:01:50.227ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 1757546723, ip_addr: 104.194.8.227, stratum: 3, ref_time: 1724817703.1109154, correction: -0.269789398 } file = sled-agent/src/rack_setup/service.rs:640
7652024-08-28T04:01:50.227ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
7662024-08-28T04:02:03.549ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 6.275666616s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f
7672024-08-28T04:02:06.317ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
7682024-08-28T04:02:06.317ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:47213
7692024-08-28T04:02:06.325ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8215 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:47213 req_id = 3a5e3ae5-2cab-4966-9e08-e2b0a163e765 response_code = 200 uri = /timesync
7702024-08-28T04:02:06.328ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 1757546723, ip_addr: 104.194.8.227, stratum: 3, ref_time: 1724817718.9608145, correction: -0.226114765 } file = sled-agent/src/rack_setup/service.rs:640
7712024-08-28T04:02:06.328ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
7722024-08-28T04:02:08.551ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:61683
7732024-08-28T04:02:08.551ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 63 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:61683 req_id = b1fbb2b7-af9f-46b3-8a02-531edf0de469 response_code = 200 uri = /baseboard
7742024-08-28T04:02:24.802ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 15.159522038s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f
7752024-08-28T04:02:30.631ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
7762024-08-28T04:02:30.631ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:47308
7772024-08-28T04:02:30.640ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8963 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:47308 req_id = 445eda6c-b363-40b9-bd59-f538ecd9ceae response_code = 200 uri = /timesync
7782024-08-28T04:02:30.643ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 1757546723, ip_addr: 104.194.8.227, stratum: 3, ref_time: 1724817743.3494267, correction: -0.160097837 } file = sled-agent/src/rack_setup/service.rs:640
7792024-08-28T04:02:30.643ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
7802024-08-28T04:02:38.507ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:54274
7812024-08-28T04:02:38.507ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 55 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:54274 req_id = 4c9ceb4c-55e0-4fef-a5eb-bd92bacb4272 response_code = 200 uri = /baseboard
7822024-08-28T04:02:54.916ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 35.975507812s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f
7832024-08-28T04:03:06.494ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
7842024-08-28T04:03:06.494ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:38380
7852024-08-28T04:03:06.503ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8515 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:38380 req_id = 0f870eb2-6f25-4357-bc32-0bfeba9fac02 response_code = 200 uri = /timesync
7862024-08-28T04:03:06.506ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 1757546723, ip_addr: 104.194.8.227, stratum: 3, ref_time: 1724817784.222648, correction: -0.062713161 } file = sled-agent/src/rack_setup/service.rs:640
7872024-08-28T04:03:06.506ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
7882024-08-28T04:03:08.459ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:65465
7892024-08-28T04:03:08.459ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 47 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:65465 req_id = 55b717a6-26c1-47f8-b2f5-8a9032038f00 response_code = 200 uri = /baseboard
7902024-08-28T04:03:38.433ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:47116
7912024-08-28T04:03:38.434ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 745 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:47116 req_id = 89cd16b6-6fe7-4fd1-93b0-b53c8514b7ae response_code = 200 uri = /baseboard
7922024-08-28T04:03:45.831ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 46.037801847s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f
7932024-08-28T04:03:53.261ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
7942024-08-28T04:03:53.261ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:43002
7952024-08-28T04:03:53.283ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 21329 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:43002 req_id = 49d987b0-cd4a-4552-8681-03777eaa3b89 response_code = 200 uri = /timesync
7962024-08-28T04:03:53.286ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: true, ref_id: 1757546723, ip_addr: 104.194.8.227, stratum: 3, ref_time: 1724817832.8249269, correction: 7.8451e-5 } file = sled-agent/src/rack_setup/service.rs:640
7972024-08-28T04:03:53.286ZINFOSledAgent (RSS): Finished setting up Internal DNS and NTP file = sled-agent/src/rack_setup/service.rs:1266
7982024-08-28T04:03:53.318ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: ae14950e-2bd4-4553-99b0-6488a03a8f3f, underlay_address: fd00:1122:3344:1::1, filesystem_pool: Some(ZpoolName { id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 4e2ba65e-422a-4b1e-b0e3-85f8149a979a, underlay_address: fd00:1122:3344:2::1, filesystem_pool: Some(ZpoolName { id: 0f906b25-a03c-4334-807e-07a343a51361 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0f906b25-a03c-4334-807e-07a343a51361 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 037b07ce-9e39-4982-8c20-c4c38f448b97, underlay_address: fd00:1122:3344:3::1, filesystem_pool: Some(ZpoolName { id: d6170dc9-a1d4-44a9-ba24-00c07a403b1e (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d6170dc9-a1d4-44a9-ba24-00c07a403b1e (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 277864c2-af7f-46e4-a527-d1f7bc0d990d, underlay_address: fd00:1122:3344:101::1b, filesystem_pool: Some(ZpoolName { id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 870956f8-434c-40b0-9f42-ba578decd127, kind: Service { id: 277864c2-af7f-46e4-a527-d1f7bc0d990d }, name: Name("ntp-277864c2-af7f-46e4-a527-d1f7bc0d990d"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 181, 11])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 0a19cb1c-82fc-4910-b47c-c6e78fe60f11, underlay_address: fd00:1122:3344:101::3, filesystem_pool: Some(ZpoolName { id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool), kind: External } } } }, OmicronZoneConfig { id: 3b3c4900-bff8-4986-b11e-76a9cc278755, underlay_address: fd00:1122:3344:101::4, filesystem_pool: Some(ZpoolName { id: 0f906b25-a03c-4334-807e-07a343a51361 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0f906b25-a03c-4334-807e-07a343a51361 (zpool), kind: External } } } }, OmicronZoneConfig { id: 488ef9e1-8ff1-4a22-9099-aacc9193d952, underlay_address: fd00:1122:3344:101::5, filesystem_pool: Some(ZpoolName { id: d6170dc9-a1d4-44a9-ba24-00c07a403b1e (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d6170dc9-a1d4-44a9-ba24-00c07a403b1e (zpool), kind: External } } } }, OmicronZoneConfig { id: 79f4b33b-000b-482c-a260-47e8c2b5e5cd, underlay_address: fd00:1122:3344:101::6, filesystem_pool: Some(ZpoolName { id: 8955d8a9-1b85-47ea-bd5c-275ef6dc0b1a (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8955d8a9-1b85-47ea-bd5c-275ef6dc0b1a (zpool), kind: External } } } }, OmicronZoneConfig { id: db3511e8-79cd-4114-b324-d82d15c2c7a2, underlay_address: fd00:1122:3344:101::7, filesystem_pool: Some(ZpoolName { id: fb7c10cb-4fd0-49dd-8c21-f83feafbd7dc (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fb7c10cb-4fd0-49dd-8c21-f83feafbd7dc (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:438 sled_address = [fd00:1122:3344:101::1]:12345
7992024-08-28T04:03:53.318ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:37810
8002024-08-28T04:03:53.318ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ae14950e-2bd4-4553-99b0-6488a03a8f3f, dataset_name: DatasetName { pool_name: ZpoolName { id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:406
8012024-08-28T04:03:53.318ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: ae14950e-2bd4-4553-99b0-6488a03a8f3f, dataset_name: DatasetName { pool_name: ZpoolName { id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool), kind: External }, kind: InternalDns }, responder: ... }
8022024-08-28T04:03:53.338ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 4e2ba65e-422a-4b1e-b0e3-85f8149a979a, dataset_name: DatasetName { pool_name: ZpoolName { id: 0f906b25-a03c-4334-807e-07a343a51361 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:406
8032024-08-28T04:03:53.338ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 4e2ba65e-422a-4b1e-b0e3-85f8149a979a, dataset_name: DatasetName { pool_name: ZpoolName { id: 0f906b25-a03c-4334-807e-07a343a51361 (zpool), kind: External }, kind: InternalDns }, responder: ... }
8042024-08-28T04:03:53.357ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 037b07ce-9e39-4982-8c20-c4c38f448b97, dataset_name: DatasetName { pool_name: ZpoolName { id: d6170dc9-a1d4-44a9-ba24-00c07a403b1e (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:406
8052024-08-28T04:03:53.357ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 037b07ce-9e39-4982-8c20-c4c38f448b97, dataset_name: DatasetName { pool_name: ZpoolName { id: d6170dc9-a1d4-44a9-ba24-00c07a403b1e (zpool), kind: External }, kind: InternalDns }, responder: ... }
8062024-08-28T04:03:53.377ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 0a19cb1c-82fc-4910-b47c-c6e78fe60f11, dataset_name: DatasetName { pool_name: ZpoolName { id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:406
8072024-08-28T04:03:53.377ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 0a19cb1c-82fc-4910-b47c-c6e78fe60f11, dataset_name: DatasetName { pool_name: ZpoolName { id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool), kind: External }, kind: CockroachDb }, responder: ... }
8082024-08-28T04:03:53.441ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 3b3c4900-bff8-4986-b11e-76a9cc278755, dataset_name: DatasetName { pool_name: ZpoolName { id: 0f906b25-a03c-4334-807e-07a343a51361 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:406
8092024-08-28T04:03:53.441ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 3b3c4900-bff8-4986-b11e-76a9cc278755, dataset_name: DatasetName { pool_name: ZpoolName { id: 0f906b25-a03c-4334-807e-07a343a51361 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
8102024-08-28T04:03:53.505ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 488ef9e1-8ff1-4a22-9099-aacc9193d952, dataset_name: DatasetName { pool_name: ZpoolName { id: d6170dc9-a1d4-44a9-ba24-00c07a403b1e (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:406
8112024-08-28T04:03:53.505ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 488ef9e1-8ff1-4a22-9099-aacc9193d952, dataset_name: DatasetName { pool_name: ZpoolName { id: d6170dc9-a1d4-44a9-ba24-00c07a403b1e (zpool), kind: External }, kind: CockroachDb }, responder: ... }
8122024-08-28T04:03:53.565ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 79f4b33b-000b-482c-a260-47e8c2b5e5cd, dataset_name: DatasetName { pool_name: ZpoolName { id: 8955d8a9-1b85-47ea-bd5c-275ef6dc0b1a (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:406
8132024-08-28T04:03:53.565ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 79f4b33b-000b-482c-a260-47e8c2b5e5cd, dataset_name: DatasetName { pool_name: ZpoolName { id: 8955d8a9-1b85-47ea-bd5c-275ef6dc0b1a (zpool), kind: External }, kind: CockroachDb }, responder: ... }
8142024-08-28T04:03:53.626ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: db3511e8-79cd-4114-b324-d82d15c2c7a2, dataset_name: DatasetName { pool_name: ZpoolName { id: fb7c10cb-4fd0-49dd-8c21-f83feafbd7dc (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:406
8152024-08-28T04:03:53.626ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: db3511e8-79cd-4114-b324-d82d15c2c7a2, dataset_name: DatasetName { pool_name: ZpoolName { id: fb7c10cb-4fd0-49dd-8c21-f83feafbd7dc (zpool), kind: External }, kind: CockroachDb }, responder: ... }
8162024-08-28T04:03:53.687ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
8172024-08-28T04:03:53.687ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/51625293-e5a8-4cc4-9640-b64176e7a48e/config/omicron-zones.json file = common/src/ledger.rs:177
8182024-08-28T04:03:53.688ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/4dad4c76-eb6c-4281-9f1d-a63fdb0117e1/config/omicron-zones.json file = common/src/ledger.rs:177
8192024-08-28T04:03:53.688ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
8202024-08-28T04:03:53.768ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
8212024-08-28T04:03:53.768ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
8222024-08-28T04:03:53.768ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
8232024-08-28T04:03:53.768ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
8242024-08-28T04:03:53.768ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
8252024-08-28T04:03:53.859ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_79f4b33b-000b-482c-a260-47e8c2b5e5cd file = illumos-utils/src/zone.rs:316
8262024-08-28T04:03:53.864ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_db3511e8-79cd-4114-b324-d82d15c2c7a2 file = illumos-utils/src/zone.rs:316
8272024-08-28T04:03:53.868ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755 file = illumos-utils/src/zone.rs:316
8282024-08-28T04:03:53.873ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_488ef9e1-8ff1-4a22-9099-aacc9193d952 file = illumos-utils/src/zone.rs:316
8292024-08-28T04:03:53.877ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11 file = illumos-utils/src/zone.rs:316
8302024-08-28T04:03:53.882ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_79f4b33b-000b-482c-a260-47e8c2b5e5cd file = illumos-utils/src/zone.rs:354
8312024-08-28T04:03:53.887ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_db3511e8-79cd-4114-b324-d82d15c2c7a2 file = illumos-utils/src/zone.rs:354
8322024-08-28T04:03:53.892ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755 file = illumos-utils/src/zone.rs:354
8332024-08-28T04:03:53.897ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_488ef9e1-8ff1-4a22-9099-aacc9193d952 file = illumos-utils/src/zone.rs:354
8342024-08-28T04:03:53.901ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11 file = illumos-utils/src/zone.rs:354
8352024-08-28T04:03:56.820ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService7'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::4'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::4]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='3b3c4900-bff8-4986-b11e-76a9cc278755'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::4]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::4]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:1::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8362024-08-28T04:03:56.821ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755
8372024-08-28T04:03:56.830ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_db3511e8-79cd-4114-b324-d82d15c2c7a2: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService6'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::7'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::7]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='db3511e8-79cd-4114-b324-d82d15c2c7a2'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::7]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::7]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:1::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8382024-08-28T04:03:56.830ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_db3511e8-79cd-4114-b324-d82d15c2c7a2 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_db3511e8-79cd-4114-b324-d82d15c2c7a2
8392024-08-28T04:03:56.917ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_79f4b33b-000b-482c-a260-47e8c2b5e5cd: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService5'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::6]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='79f4b33b-000b-482c-a260-47e8c2b5e5cd'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::6]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::6]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:1::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8402024-08-28T04:03:56.917ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_79f4b33b-000b-482c-a260-47e8c2b5e5cd zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_79f4b33b-000b-482c-a260-47e8c2b5e5cd
8412024-08-28T04:03:56.923ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService9'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::3'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::3]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='0a19cb1c-82fc-4910-b47c-c6e78fe60f11'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::3]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::3]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:1::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8422024-08-28T04:03:56.924ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11
8432024-08-28T04:03:56.965ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755") failed: Property not found. retry in 40.709584ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755
8442024-08-28T04:03:56.973ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_db3511e8-79cd-4114-b324-d82d15c2c7a2") failed: Property not found. retry in 56.111315ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_db3511e8-79cd-4114-b324-d82d15c2c7a2
8452024-08-28T04:03:56.980ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_488ef9e1-8ff1-4a22-9099-aacc9193d952: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService8'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::5]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='488ef9e1-8ff1-4a22-9099-aacc9193d952'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::5]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::5]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:1::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8462024-08-28T04:03:56.980ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_488ef9e1-8ff1-4a22-9099-aacc9193d952 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_488ef9e1-8ff1-4a22-9099-aacc9193d952
8472024-08-28T04:03:57.015ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755") failed: Property not found. retry in 133.979607ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755
8482024-08-28T04:03:57.192ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_db3511e8-79cd-4114-b324-d82d15c2c7a2") failed: Property not found. retry in 102.098854ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_db3511e8-79cd-4114-b324-d82d15c2c7a2
8492024-08-28T04:03:57.242ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755") failed: Property not found. retry in 149.862793ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755
8502024-08-28T04:03:57.329ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_79f4b33b-000b-482c-a260-47e8c2b5e5cd") failed: Property not found. retry in 37.916703ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_79f4b33b-000b-482c-a260-47e8c2b5e5cd
8512024-08-28T04:03:57.438ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_db3511e8-79cd-4114-b324-d82d15c2c7a2") failed: Property not found. retry in 260.483505ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_db3511e8-79cd-4114-b324-d82d15c2c7a2
8522024-08-28T04:03:57.709ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755") failed: Property not found. retry in 357.925558ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755
8532024-08-28T04:03:57.740ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11") failed: Property not found. retry in 42.400389ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11
8542024-08-28T04:03:57.763ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_79f4b33b-000b-482c-a260-47e8c2b5e5cd") failed: Property not found. retry in 125.621ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_79f4b33b-000b-482c-a260-47e8c2b5e5cd
8552024-08-28T04:03:57.806ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_db3511e8-79cd-4114-b324-d82d15c2c7a2") failed: Property not found. retry in 421.292304ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_db3511e8-79cd-4114-b324-d82d15c2c7a2
8562024-08-28T04:03:58.030ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_488ef9e1-8ff1-4a22-9099-aacc9193d952") failed: Property not found. retry in 31.793704ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_488ef9e1-8ff1-4a22-9099-aacc9193d952
8572024-08-28T04:03:58.110ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11") failed: Property not found. retry in 129.030076ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11
8582024-08-28T04:03:58.260ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_79f4b33b-000b-482c-a260-47e8c2b5e5cd") failed: Property not found. retry in 107.806334ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_79f4b33b-000b-482c-a260-47e8c2b5e5cd
8592024-08-28T04:03:58.366ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_db3511e8-79cd-4114-b324-d82d15c2c7a2") failed: Property not found. retry in 655.932004ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_db3511e8-79cd-4114-b324-d82d15c2c7a2
8602024-08-28T04:03:58.705ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755") failed: Property not found. retry in 1.087529959s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755
8612024-08-28T04:03:58.888ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_488ef9e1-8ff1-4a22-9099-aacc9193d952") failed: Property not found. retry in 105.17439ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_488ef9e1-8ff1-4a22-9099-aacc9193d952
8622024-08-28T04:03:59.199ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11") failed: Property not found. retry in 101.066833ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11
8632024-08-28T04:03:59.223ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_79f4b33b-000b-482c-a260-47e8c2b5e5cd") failed: Property not found. retry in 364.924803ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_79f4b33b-000b-482c-a260-47e8c2b5e5cd
8642024-08-28T04:03:59.242ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_db3511e8-79cd-4114-b324-d82d15c2c7a2") failed: Property not found. retry in 1.002158301s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_db3511e8-79cd-4114-b324-d82d15c2c7a2
8652024-08-28T04:03:59.288ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_488ef9e1-8ff1-4a22-9099-aacc9193d952") failed: Property not found. retry in 115.306562ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_488ef9e1-8ff1-4a22-9099-aacc9193d952
8662024-08-28T04:03:59.328ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11") failed: Property not found. retry in 502.35057ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11
8672024-08-28T04:03:59.417ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_488ef9e1-8ff1-4a22-9099-aacc9193d952") failed: Property not found. retry in 273.810464ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_488ef9e1-8ff1-4a22-9099-aacc9193d952
8682024-08-28T04:03:59.599ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_79f4b33b-000b-482c-a260-47e8c2b5e5cd") failed: Property not found. retry in 658.615235ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_79f4b33b-000b-482c-a260-47e8c2b5e5cd
8692024-08-28T04:03:59.702ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_488ef9e1-8ff1-4a22-9099-aacc9193d952") failed: Property not found. retry in 958.803142ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_488ef9e1-8ff1-4a22-9099-aacc9193d952
8702024-08-28T04:03:59.804ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755") failed: Property not found. retry in 684.617426ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755
8712024-08-28T04:03:59.842ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11") failed: Property not found. retry in 630.381433ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11
8722024-08-28T04:04:00.254ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_db3511e8-79cd-4114-b324-d82d15c2c7a2") failed: Property not found. retry in 1.203763002s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_db3511e8-79cd-4114-b324-d82d15c2c7a2
8732024-08-28T04:04:00.266ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_79f4b33b-000b-482c-a260-47e8c2b5e5cd") failed: Property not found. retry in 658.486144ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_79f4b33b-000b-482c-a260-47e8c2b5e5cd
8742024-08-28T04:04:00.482ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11") failed: Property not found. retry in 1.372528045s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11
8752024-08-28T04:04:00.497ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755") failed: Property not found. retry in 1.375987658s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755
8762024-08-28T04:04:00.671ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_488ef9e1-8ff1-4a22-9099-aacc9193d952") failed: Property not found. retry in 1.478303584s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_488ef9e1-8ff1-4a22-9099-aacc9193d952
8772024-08-28T04:04:00.933ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_79f4b33b-000b-482c-a260-47e8c2b5e5cd") failed: Property not found. retry in 1.365880777s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_79f4b33b-000b-482c-a260-47e8c2b5e5cd
8782024-08-28T04:04:01.469ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_db3511e8-79cd-4114-b324-d82d15c2c7a2") failed: Property not found. retry in 1.287653379s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_db3511e8-79cd-4114-b324-d82d15c2c7a2
8792024-08-28T04:04:01.865ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11") failed: Property not found. retry in 519.949421ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11
8802024-08-28T04:04:01.883ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755") failed: Property not found. retry in 1.324616344s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755
8812024-08-28T04:04:02.166ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_488ef9e1-8ff1-4a22-9099-aacc9193d952") failed: Property not found. retry in 1.244115667s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_488ef9e1-8ff1-4a22-9099-aacc9193d952
8822024-08-28T04:04:02.313ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_79f4b33b-000b-482c-a260-47e8c2b5e5cd") failed: Property not found. retry in 1.190328442s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_79f4b33b-000b-482c-a260-47e8c2b5e5cd
8832024-08-28T04:04:02.398ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11") failed: Property not found. retry in 944.405779ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11
8842024-08-28T04:04:02.766ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_db3511e8-79cd-4114-b324-d82d15c2c7a2") failed: Property not found. retry in 1.090617756s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_db3511e8-79cd-4114-b324-d82d15c2c7a2
8852024-08-28T04:04:03.218ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755") failed: Property not found. retry in 563.44138ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755
8862024-08-28T04:04:03.351ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11") failed: Property not found. retry in 1.213772617s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11
8872024-08-28T04:04:03.419ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_488ef9e1-8ff1-4a22-9099-aacc9193d952") failed: Property not found. retry in 612.01577ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_488ef9e1-8ff1-4a22-9099-aacc9193d952
8882024-08-28T04:04:03.513ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_79f4b33b-000b-482c-a260-47e8c2b5e5cd") failed: Property not found. retry in 1.258959152s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_79f4b33b-000b-482c-a260-47e8c2b5e5cd
8892024-08-28T04:04:03.789ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755") failed: Property not found. retry in 947.721387ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755
8902024-08-28T04:04:03.865ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_db3511e8-79cd-4114-b324-d82d15c2c7a2") failed: Property not found. retry in 1.422985994s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_db3511e8-79cd-4114-b324-d82d15c2c7a2
8912024-08-28T04:04:04.039ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_488ef9e1-8ff1-4a22-9099-aacc9193d952") failed: Property not found. retry in 1.119589047s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_488ef9e1-8ff1-4a22-9099-aacc9193d952
8922024-08-28T04:04:04.575ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11") failed: Property not found. retry in 568.054981ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11
8932024-08-28T04:04:04.746ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755") failed: Property not found. retry in 948.644567ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755
8942024-08-28T04:04:04.780ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_79f4b33b-000b-482c-a260-47e8c2b5e5cd") failed: Property not found. retry in 1.201685864s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_79f4b33b-000b-482c-a260-47e8c2b5e5cd
8952024-08-28T04:04:05.154ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11") failed: Property not found. retry in 805.918664ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11
8962024-08-28T04:04:05.168ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_488ef9e1-8ff1-4a22-9099-aacc9193d952") failed: Property not found. retry in 1.009572542s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_488ef9e1-8ff1-4a22-9099-aacc9193d952
8972024-08-28T04:04:05.706ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755") failed: Property not found. retry in 1.181079364s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755
8982024-08-28T04:04:05.972ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11") failed: Property not found. retry in 1.209994903s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11
8992024-08-28T04:04:06.899ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755") failed: Property not found. retry in 1.163669605s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755
9002024-08-28T04:04:07.194ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11") failed: Property not found. retry in 957.491378ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11
9012024-08-28T04:04:08.075ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755") failed: Property not found. retry in 1.395169087s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755
9022024-08-28T04:04:08.162ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11") failed: Property not found. retry in 1.032542882s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11
9032024-08-28T04:04:08.470ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:46692
9042024-08-28T04:04:08.470ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 82 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:46692 req_id = 51cad66a-085d-4960-beb6-3e3666bc51a3 response_code = 200 uri = /baseboard
9052024-08-28T04:04:09.205ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11") failed: Property not found. retry in 1.327473847s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11
9062024-08-28T04:04:09.526ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755") failed: Property not found. retry in 622.372476ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755
9072024-08-28T04:04:10.160ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755") failed: Property not found. retry in 1.443816544s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755
9082024-08-28T04:04:10.545ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11") failed: Property not found. retry in 1.309750125s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11
9092024-08-28T04:04:11.612ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755") failed: Property not found. retry in 1.34074508s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755
9102024-08-28T04:04:11.866ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11") failed: Property not found. retry in 621.968173ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11
9112024-08-28T04:04:12.497ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11") failed: Property not found. retry in 614.760439ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11
9122024-08-28T04:04:12.961ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755") failed: Property not found. retry in 545.519881ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755
9132024-08-28T04:04:13.119ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11") failed: Property not found. retry in 1.176857225s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11
9142024-08-28T04:04:13.514ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755") failed: Property not found. retry in 1.117181318s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755
9152024-08-28T04:04:14.305ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11") failed: Property not found. retry in 1.438788938s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11
9162024-08-28T04:04:14.640ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755") failed: Property not found. retry in 1.409277366s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755
9172024-08-28T04:04:16.072ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_cockroachdb_79f4b33b-000b-482c-a260-47e8c2b5e5cd
9182024-08-28T04:04:16.072ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_cockroachdb_db3511e8-79cd-4114-b324-d82d15c2c7a2
9192024-08-28T04:04:16.072ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_cockroachdb_3b3c4900-bff8-4986-b11e-76a9cc278755
9202024-08-28T04:04:16.072ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_cockroachdb_488ef9e1-8ff1-4a22-9099-aacc9193d952
9212024-08-28T04:04:16.072ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_cockroachdb_0a19cb1c-82fc-4910-b47c-c6e78fe60f11
9222024-08-28T04:04:16.072ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/51625293-e5a8-4cc4-9640-b64176e7a48e/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
9232024-08-28T04:04:16.073ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/4dad4c76-eb6c-4281-9f1d-a63fdb0117e1/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
9242024-08-28T04:04:16.073ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 22754850 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:37810 req_id = aa5cc73c-bf13-4355-8d6b-d615ffd7c3e7 response_code = 204 uri = /omicron-zones
9252024-08-28T04:04:16.108ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:50893
9262024-08-28T04:04:16.139ZINFOSledAgent (ServiceManager): Initializing CRDB Cluster - sending request to [fd00:1122:3344:101::3]:32221 file = sled-agent/src/services.rs:3681
9272024-08-28T04:04:26.456ZINFOSledAgent (ServiceManager): Formatting CRDB file = sled-agent/src/services.rs:3699
9282024-08-28T04:04:35.739ZINFOSledAgent (ServiceManager): Formatting CRDB - Completed file = sled-agent/src/services.rs:3722
9292024-08-28T04:04:35.739ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 19630913 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:50893 req_id = 12d427db-c370-48fe-811a-58647148f28a response_code = 204 uri = /cockroachdb
9302024-08-28T04:04:35.805ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(5), zones: [OmicronZoneConfig { id: ae14950e-2bd4-4553-99b0-6488a03a8f3f, underlay_address: fd00:1122:3344:1::1, filesystem_pool: Some(ZpoolName { id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 4e2ba65e-422a-4b1e-b0e3-85f8149a979a, underlay_address: fd00:1122:3344:2::1, filesystem_pool: Some(ZpoolName { id: 0f906b25-a03c-4334-807e-07a343a51361 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0f906b25-a03c-4334-807e-07a343a51361 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 037b07ce-9e39-4982-8c20-c4c38f448b97, underlay_address: fd00:1122:3344:3::1, filesystem_pool: Some(ZpoolName { id: d6170dc9-a1d4-44a9-ba24-00c07a403b1e (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d6170dc9-a1d4-44a9-ba24-00c07a403b1e (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 277864c2-af7f-46e4-a527-d1f7bc0d990d, underlay_address: fd00:1122:3344:101::1b, filesystem_pool: Some(ZpoolName { id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 870956f8-434c-40b0-9f42-ba578decd127, kind: Service { id: 277864c2-af7f-46e4-a527-d1f7bc0d990d }, name: Name("ntp-277864c2-af7f-46e4-a527-d1f7bc0d990d"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 181, 11])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 0a19cb1c-82fc-4910-b47c-c6e78fe60f11, underlay_address: fd00:1122:3344:101::3, filesystem_pool: Some(ZpoolName { id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool), kind: External } } } }, OmicronZoneConfig { id: 3b3c4900-bff8-4986-b11e-76a9cc278755, underlay_address: fd00:1122:3344:101::4, filesystem_pool: Some(ZpoolName { id: 0f906b25-a03c-4334-807e-07a343a51361 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0f906b25-a03c-4334-807e-07a343a51361 (zpool), kind: External } } } }, OmicronZoneConfig { id: 488ef9e1-8ff1-4a22-9099-aacc9193d952, underlay_address: fd00:1122:3344:101::5, filesystem_pool: Some(ZpoolName { id: d6170dc9-a1d4-44a9-ba24-00c07a403b1e (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d6170dc9-a1d4-44a9-ba24-00c07a403b1e (zpool), kind: External } } } }, OmicronZoneConfig { id: 79f4b33b-000b-482c-a260-47e8c2b5e5cd, underlay_address: fd00:1122:3344:101::6, filesystem_pool: Some(ZpoolName { id: 8955d8a9-1b85-47ea-bd5c-275ef6dc0b1a (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8955d8a9-1b85-47ea-bd5c-275ef6dc0b1a (zpool), kind: External } } } }, OmicronZoneConfig { id: db3511e8-79cd-4114-b324-d82d15c2c7a2, underlay_address: fd00:1122:3344:101::7, filesystem_pool: Some(ZpoolName { id: fb7c10cb-4fd0-49dd-8c21-f83feafbd7dc (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fb7c10cb-4fd0-49dd-8c21-f83feafbd7dc (zpool), kind: External } } } }, OmicronZoneConfig { id: 4eb364e4-14a2-46c4-b173-eb2890920e8f, underlay_address: fd00:1122:3344:101::8, filesystem_pool: Some(ZpoolName { id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool), kind: External }), zone_type: ExternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool), kind: External } }, http_address: [fd00:1122:3344:101::8]:5353, dns_address: 10.151.2.164:53, nic: NetworkInterface { id: 4b747939-586b-48cb-a488-7f7f972db309, kind: Service { id: 4eb364e4-14a2-46c4-b173-eb2890920e8f }, name: Name("external-dns-4eb364e4-14a2-46c4-b173-eb2890920e8f"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 246, 113])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } } }, OmicronZoneConfig { id: 94b35d7a-af71-4078-a57c-874ff9a27068, underlay_address: fd00:1122:3344:101::9, filesystem_pool: Some(ZpoolName { id: 0f906b25-a03c-4334-807e-07a343a51361 (zpool), kind: External }), zone_type: ExternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0f906b25-a03c-4334-807e-07a343a51361 (zpool), kind: External } }, http_address: [fd00:1122:3344:101::9]:5353, dns_address: 10.151.2.165:53, nic: NetworkInterface { id: a3e0f17f-d79d-4cc0-ba0f-997e844ad5f3, kind: Service { id: 94b35d7a-af71-4078-a57c-874ff9a27068 }, name: Name("external-dns-94b35d7a-af71-4078-a57c-874ff9a27068"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 165, 179])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } } }, OmicronZoneConfig { id: 4c8a8ded-ced6-466f-8626-7fd39c37d020, underlay_address: fd00:1122:3344:101::a, filesystem_pool: Some(ZpoolName { id: d6170dc9-a1d4-44a9-ba24-00c07a403b1e (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::a]:12221, external_ip: 10.151.2.166, nic: NetworkInterface { id: 1a83b6d1-627c-41ca-8268-a29f26800e91, kind: Service { id: 4c8a8ded-ced6-466f-8626-7fd39c37d020 }, name: Name("nexus-4c8a8ded-ced6-466f-8626-7fd39c37d020"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 220])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: 1c6ee351-cab6-4c23-8658-c31bac40b40a, underlay_address: fd00:1122:3344:101::b, filesystem_pool: Some(ZpoolName { id: 8955d8a9-1b85-47ea-bd5c-275ef6dc0b1a (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::b]:12221, external_ip: 10.151.2.167, nic: NetworkInterface { id: 0cddb53a-efd3-4f89-93d7-81b182af0cdb, kind: Service { id: 1c6ee351-cab6-4c23-8658-c31bac40b40a }, name: Name("nexus-1c6ee351-cab6-4c23-8658-c31bac40b40a"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 208, 245])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: bda2ed8e-2ea5-41eb-824b-d14527b76b8b, underlay_address: fd00:1122:3344:101::c, filesystem_pool: Some(ZpoolName { id: 8955d8a9-1b85-47ea-bd5c-275ef6dc0b1a (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::c]:12221, external_ip: 10.151.2.168, nic: NetworkInterface { id: 34a49c6e-26fe-4abd-8702-695d279dbdca, kind: Service { id: bda2ed8e-2ea5-41eb-824b-d14527b76b8b }, name: Name("nexus-bda2ed8e-2ea5-41eb-824b-d14527b76b8b"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 210, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: 21378de4-15c3-48e2-b919-b1cc2363c4a6, underlay_address: fd00:1122:3344:101::d, filesystem_pool: Some(ZpoolName { id: ba3c519e-f98a-4ee2-a345-a21845f5f7a4 (zpool), kind: External }), zone_type: Oximeter { address: [fd00:1122:3344:101::d]:12223 } }, OmicronZoneConfig { id: f62eaa35-1217-424b-a535-2aa55c06fbad, underlay_address: fd00:1122:3344:101::e, filesystem_pool: Some(ZpoolName { id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool), kind: External }), zone_type: Clickhouse { address: [fd00:1122:3344:101::e]:8123, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool), kind: External } } } }, OmicronZoneConfig { id: 83266762-372c-41cd-a9dd-eee1b2c38bb6, underlay_address: fd00:1122:3344:101::f, filesystem_pool: Some(ZpoolName { id: ba3c519e-f98a-4ee2-a345-a21845f5f7a4 (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::f]:17000 } }, OmicronZoneConfig { id: 1d429812-8d5a-47a0-98f7-1f1f5ef6e285, underlay_address: fd00:1122:3344:101::10, filesystem_pool: Some(ZpoolName { id: ba3c519e-f98a-4ee2-a345-a21845f5f7a4 (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::10]:17000 } }, OmicronZoneConfig { id: d56672e9-8c1d-4f9a-a4b1-49fa3e1cfbe0, underlay_address: fd00:1122:3344:101::11, filesystem_pool: Some(ZpoolName { id: fb7c10cb-4fd0-49dd-8c21-f83feafbd7dc (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::11]:17000 } }, OmicronZoneConfig { id: 4b839573-b270-4df0-b615-f92964eededd, underlay_address: fd00:1122:3344:101::12, filesystem_pool: Some(ZpoolName { id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::12]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool), kind: External } } } }, OmicronZoneConfig { id: 24b1ac7b-c1d1-440a-a4e4-220c7ce34410, underlay_address: fd00:1122:3344:101::13, filesystem_pool: Some(ZpoolName { id: 0f906b25-a03c-4334-807e-07a343a51361 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::13]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0f906b25-a03c-4334-807e-07a343a51361 (zpool), kind: External } } } }, OmicronZoneConfig { id: d6ce85cc-4f25-4b5e-b0c0-a434368f63d4, underlay_address: fd00:1122:3344:101::14, filesystem_pool: Some(ZpoolName { id: d6170dc9-a1d4-44a9-ba24-00c07a403b1e (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::14]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d6170dc9-a1d4-44a9-ba24-00c07a403b1e (zpool), kind: External } } } }, OmicronZoneConfig { id: d073ae9b-afee-401e-bafb-45a05ad31f0d, underlay_address: fd00:1122:3344:101::15, filesystem_pool: Some(ZpoolName { id: 8955d8a9-1b85-47ea-bd5c-275ef6dc0b1a (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::15]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8955d8a9-1b85-47ea-bd5c-275ef6dc0b1a (zpool), kind: External } } } }, OmicronZoneConfig { id: ff9eaaaf-6eec-47be-bc80-11d83c51090f, underlay_address: fd00:1122:3344:101::16, filesystem_pool: Some(ZpoolName { id: fb7c10cb-4fd0-49dd-8c21-f83feafbd7dc (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::16]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fb7c10cb-4fd0-49dd-8c21-f83feafbd7dc (zpool), kind: External } } } }, OmicronZoneConfig { id: 7e16ed01-11ff-45eb-8869-6ed06d9b0e70, underlay_address: fd00:1122:3344:101::17, filesystem_pool: Some(ZpoolName { id: ba3c519e-f98a-4ee2-a345-a21845f5f7a4 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::17]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ba3c519e-f98a-4ee2-a345-a21845f5f7a4 (zpool), kind: External } } } }, OmicronZoneConfig { id: 7f6ebc53-fc17-488d-9b99-8dc983f4efec, underlay_address: fd00:1122:3344:101::18, filesystem_pool: Some(ZpoolName { id: 0913dfa8-99f2-4518-8d37-827ea32c30ba (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::18]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0913dfa8-99f2-4518-8d37-827ea32c30ba (zpool), kind: External } } } }, OmicronZoneConfig { id: a8d89411-e3d4-4290-8995-191bd8cdd564, underlay_address: fd00:1122:3344:101::19, filesystem_pool: Some(ZpoolName { id: ed7027bd-3cf9-4764-bc15-404c842ab02a (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::19]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ed7027bd-3cf9-4764-bc15-404c842ab02a (zpool), kind: External } } } }, OmicronZoneConfig { id: d4f01887-9496-477a-9ed8-f93bf14fea73, underlay_address: fd00:1122:3344:101::1a, filesystem_pool: Some(ZpoolName { id: e3fa46c7-4f3a-45f7-afd8-b7799f191587 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::1a]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: e3fa46c7-4f3a-45f7-afd8-b7799f191587 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:438 sled_address = [fd00:1122:3344:101::1]:12345
9312024-08-28T04:04:35.805ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:32833
9322024-08-28T04:04:35.806ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ae14950e-2bd4-4553-99b0-6488a03a8f3f, dataset_name: DatasetName { pool_name: ZpoolName { id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:406
9332024-08-28T04:04:35.806ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: ae14950e-2bd4-4553-99b0-6488a03a8f3f, dataset_name: DatasetName { pool_name: ZpoolName { id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool), kind: External }, kind: InternalDns }, responder: ... }
9342024-08-28T04:04:35.830ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 4e2ba65e-422a-4b1e-b0e3-85f8149a979a, dataset_name: DatasetName { pool_name: ZpoolName { id: 0f906b25-a03c-4334-807e-07a343a51361 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:406
9352024-08-28T04:04:35.830ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 4e2ba65e-422a-4b1e-b0e3-85f8149a979a, dataset_name: DatasetName { pool_name: ZpoolName { id: 0f906b25-a03c-4334-807e-07a343a51361 (zpool), kind: External }, kind: InternalDns }, responder: ... }
9362024-08-28T04:04:35.850ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 037b07ce-9e39-4982-8c20-c4c38f448b97, dataset_name: DatasetName { pool_name: ZpoolName { id: d6170dc9-a1d4-44a9-ba24-00c07a403b1e (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:406
9372024-08-28T04:04:35.850ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 037b07ce-9e39-4982-8c20-c4c38f448b97, dataset_name: DatasetName { pool_name: ZpoolName { id: d6170dc9-a1d4-44a9-ba24-00c07a403b1e (zpool), kind: External }, kind: InternalDns }, responder: ... }
9382024-08-28T04:04:35.871ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 0a19cb1c-82fc-4910-b47c-c6e78fe60f11, dataset_name: DatasetName { pool_name: ZpoolName { id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:406
9392024-08-28T04:04:35.871ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 0a19cb1c-82fc-4910-b47c-c6e78fe60f11, dataset_name: DatasetName { pool_name: ZpoolName { id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool), kind: External }, kind: CockroachDb }, responder: ... }
9402024-08-28T04:04:35.891ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 3b3c4900-bff8-4986-b11e-76a9cc278755, dataset_name: DatasetName { pool_name: ZpoolName { id: 0f906b25-a03c-4334-807e-07a343a51361 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:406
9412024-08-28T04:04:35.891ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 3b3c4900-bff8-4986-b11e-76a9cc278755, dataset_name: DatasetName { pool_name: ZpoolName { id: 0f906b25-a03c-4334-807e-07a343a51361 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
9422024-08-28T04:04:35.910ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 488ef9e1-8ff1-4a22-9099-aacc9193d952, dataset_name: DatasetName { pool_name: ZpoolName { id: d6170dc9-a1d4-44a9-ba24-00c07a403b1e (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:406
9432024-08-28T04:04:35.910ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 488ef9e1-8ff1-4a22-9099-aacc9193d952, dataset_name: DatasetName { pool_name: ZpoolName { id: d6170dc9-a1d4-44a9-ba24-00c07a403b1e (zpool), kind: External }, kind: CockroachDb }, responder: ... }
9442024-08-28T04:04:35.930ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 79f4b33b-000b-482c-a260-47e8c2b5e5cd, dataset_name: DatasetName { pool_name: ZpoolName { id: 8955d8a9-1b85-47ea-bd5c-275ef6dc0b1a (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:406
9452024-08-28T04:04:35.930ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 79f4b33b-000b-482c-a260-47e8c2b5e5cd, dataset_name: DatasetName { pool_name: ZpoolName { id: 8955d8a9-1b85-47ea-bd5c-275ef6dc0b1a (zpool), kind: External }, kind: CockroachDb }, responder: ... }
9462024-08-28T04:04:35.951ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: db3511e8-79cd-4114-b324-d82d15c2c7a2, dataset_name: DatasetName { pool_name: ZpoolName { id: fb7c10cb-4fd0-49dd-8c21-f83feafbd7dc (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:406
9472024-08-28T04:04:35.951ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: db3511e8-79cd-4114-b324-d82d15c2c7a2, dataset_name: DatasetName { pool_name: ZpoolName { id: fb7c10cb-4fd0-49dd-8c21-f83feafbd7dc (zpool), kind: External }, kind: CockroachDb }, responder: ... }
9482024-08-28T04:04:35.969ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 4eb364e4-14a2-46c4-b173-eb2890920e8f, dataset_name: DatasetName { pool_name: ZpoolName { id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool), kind: External }, kind: ExternalDns }, responder: ... }) file = sled-storage/src/manager.rs:406
9492024-08-28T04:04:35.969ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 4eb364e4-14a2-46c4-b173-eb2890920e8f, dataset_name: DatasetName { pool_name: ZpoolName { id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool), kind: External }, kind: ExternalDns }, responder: ... }
9502024-08-28T04:04:36.089ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 94b35d7a-af71-4078-a57c-874ff9a27068, dataset_name: DatasetName { pool_name: ZpoolName { id: 0f906b25-a03c-4334-807e-07a343a51361 (zpool), kind: External }, kind: ExternalDns }, responder: ... }) file = sled-storage/src/manager.rs:406
9512024-08-28T04:04:36.089ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 94b35d7a-af71-4078-a57c-874ff9a27068, dataset_name: DatasetName { pool_name: ZpoolName { id: 0f906b25-a03c-4334-807e-07a343a51361 (zpool), kind: External }, kind: ExternalDns }, responder: ... }
9522024-08-28T04:04:36.196ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f62eaa35-1217-424b-a535-2aa55c06fbad, dataset_name: DatasetName { pool_name: ZpoolName { id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool), kind: External }, kind: Clickhouse }, responder: ... }) file = sled-storage/src/manager.rs:406
9532024-08-28T04:04:36.196ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: f62eaa35-1217-424b-a535-2aa55c06fbad, dataset_name: DatasetName { pool_name: ZpoolName { id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool), kind: External }, kind: Clickhouse }, responder: ... }
9542024-08-28T04:04:36.301ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 4b839573-b270-4df0-b615-f92964eededd, dataset_name: DatasetName { pool_name: ZpoolName { id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:406
9552024-08-28T04:04:36.301ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 4b839573-b270-4df0-b615-f92964eededd, dataset_name: DatasetName { pool_name: ZpoolName { id: 57750eb9-c738-4eb8-89f2-c5eab164d34c (zpool), kind: External }, kind: Crucible }, responder: ... }
9562024-08-28T04:04:36.398ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 24b1ac7b-c1d1-440a-a4e4-220c7ce34410, dataset_name: DatasetName { pool_name: ZpoolName { id: 0f906b25-a03c-4334-807e-07a343a51361 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:406
9572024-08-28T04:04:36.398ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 24b1ac7b-c1d1-440a-a4e4-220c7ce34410, dataset_name: DatasetName { pool_name: ZpoolName { id: 0f906b25-a03c-4334-807e-07a343a51361 (zpool), kind: External }, kind: Crucible }, responder: ... }
9582024-08-28T04:04:36.487ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: d6ce85cc-4f25-4b5e-b0c0-a434368f63d4, dataset_name: DatasetName { pool_name: ZpoolName { id: d6170dc9-a1d4-44a9-ba24-00c07a403b1e (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:406
9592024-08-28T04:04:36.487ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: d6ce85cc-4f25-4b5e-b0c0-a434368f63d4, dataset_name: DatasetName { pool_name: ZpoolName { id: d6170dc9-a1d4-44a9-ba24-00c07a403b1e (zpool), kind: External }, kind: Crucible }, responder: ... }
9602024-08-28T04:04:36.586ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: d073ae9b-afee-401e-bafb-45a05ad31f0d, dataset_name: DatasetName { pool_name: ZpoolName { id: 8955d8a9-1b85-47ea-bd5c-275ef6dc0b1a (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:406
9612024-08-28T04:04:36.586ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: d073ae9b-afee-401e-bafb-45a05ad31f0d, dataset_name: DatasetName { pool_name: ZpoolName { id: 8955d8a9-1b85-47ea-bd5c-275ef6dc0b1a (zpool), kind: External }, kind: Crucible }, responder: ... }
9622024-08-28T04:04:36.705ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ff9eaaaf-6eec-47be-bc80-11d83c51090f, dataset_name: DatasetName { pool_name: ZpoolName { id: fb7c10cb-4fd0-49dd-8c21-f83feafbd7dc (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:406
9632024-08-28T04:04:36.705ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: ff9eaaaf-6eec-47be-bc80-11d83c51090f, dataset_name: DatasetName { pool_name: ZpoolName { id: fb7c10cb-4fd0-49dd-8c21-f83feafbd7dc (zpool), kind: External }, kind: Crucible }, responder: ... }
9642024-08-28T04:04:36.792ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 7e16ed01-11ff-45eb-8869-6ed06d9b0e70, dataset_name: DatasetName { pool_name: ZpoolName { id: ba3c519e-f98a-4ee2-a345-a21845f5f7a4 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:406
9652024-08-28T04:04:36.792ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 7e16ed01-11ff-45eb-8869-6ed06d9b0e70, dataset_name: DatasetName { pool_name: ZpoolName { id: ba3c519e-f98a-4ee2-a345-a21845f5f7a4 (zpool), kind: External }, kind: Crucible }, responder: ... }
9662024-08-28T04:04:36.865ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 7f6ebc53-fc17-488d-9b99-8dc983f4efec, dataset_name: DatasetName { pool_name: ZpoolName { id: 0913dfa8-99f2-4518-8d37-827ea32c30ba (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:406
9672024-08-28T04:04:36.865ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 7f6ebc53-fc17-488d-9b99-8dc983f4efec, dataset_name: DatasetName { pool_name: ZpoolName { id: 0913dfa8-99f2-4518-8d37-827ea32c30ba (zpool), kind: External }, kind: Crucible }, responder: ... }
9682024-08-28T04:04:36.954ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: a8d89411-e3d4-4290-8995-191bd8cdd564, dataset_name: DatasetName { pool_name: ZpoolName { id: ed7027bd-3cf9-4764-bc15-404c842ab02a (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:406
9692024-08-28T04:04:36.954ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: a8d89411-e3d4-4290-8995-191bd8cdd564, dataset_name: DatasetName { pool_name: ZpoolName { id: ed7027bd-3cf9-4764-bc15-404c842ab02a (zpool), kind: External }, kind: Crucible }, responder: ... }
9702024-08-28T04:04:37.049ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: d4f01887-9496-477a-9ed8-f93bf14fea73, dataset_name: DatasetName { pool_name: ZpoolName { id: e3fa46c7-4f3a-45f7-afd8-b7799f191587 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:406
9712024-08-28T04:04:37.049ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: d4f01887-9496-477a-9ed8-f93bf14fea73, dataset_name: DatasetName { pool_name: ZpoolName { id: e3fa46c7-4f3a-45f7-afd8-b7799f191587 (zpool), kind: External }, kind: Crucible }, responder: ... }
9722024-08-28T04:04:37.138ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
9732024-08-28T04:04:37.138ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/51625293-e5a8-4cc4-9640-b64176e7a48e/config/omicron-zones.json file = common/src/ledger.rs:177
9742024-08-28T04:04:37.139ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/4dad4c76-eb6c-4281-9f1d-a63fdb0117e1/config/omicron-zones.json file = common/src/ledger.rs:177
9752024-08-28T04:04:37.139ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
9762024-08-28T04:04:37.294ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
9772024-08-28T04:04:37.295ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
9782024-08-28T04:04:37.295ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
9792024-08-28T04:04:37.295ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
9802024-08-28T04:04:37.295ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
9812024-08-28T04:04:37.295ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
9822024-08-28T04:04:37.305ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
9832024-08-28T04:04:37.342ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
9842024-08-28T04:04:37.342ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
9852024-08-28T04:04:37.342ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
9862024-08-28T04:04:37.493ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
9872024-08-28T04:04:37.511ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
9882024-08-28T04:04:37.511ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
9892024-08-28T04:04:37.511ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
9902024-08-28T04:04:37.511ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
9912024-08-28T04:04:37.511ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
9922024-08-28T04:04:37.513ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
9932024-08-28T04:04:37.536ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
9942024-08-28T04:04:37.536ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
9952024-08-28T04:04:37.536ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_d6ce85cc-4f25-4b5e-b0c0-a434368f63d4 file = illumos-utils/src/zone.rs:316
9962024-08-28T04:04:37.542ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_d4f01887-9496-477a-9ed8-f93bf14fea73 file = illumos-utils/src/zone.rs:316
9972024-08-28T04:04:37.547ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_4b839573-b270-4df0-b615-f92964eededd file = illumos-utils/src/zone.rs:316
9982024-08-28T04:04:37.552ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_oximeter_21378de4-15c3-48e2-b919-b1cc2363c4a6 file = illumos-utils/src/zone.rs:316
9992024-08-28T04:04:37.557ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_d073ae9b-afee-401e-bafb-45a05ad31f0d file = illumos-utils/src/zone.rs:316
10002024-08-28T04:04:37.563ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_ff9eaaaf-6eec-47be-bc80-11d83c51090f file = illumos-utils/src/zone.rs:316
10012024-08-28T04:04:37.568ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_7e16ed01-11ff-45eb-8869-6ed06d9b0e70 file = illumos-utils/src/zone.rs:316
10022024-08-28T04:04:37.573ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_a8d89411-e3d4-4290-8995-191bd8cdd564 file = illumos-utils/src/zone.rs:316
10032024-08-28T04:04:37.697ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_d6ce85cc-4f25-4b5e-b0c0-a434368f63d4 file = illumos-utils/src/zone.rs:354
10042024-08-28T04:04:37.701ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_d4f01887-9496-477a-9ed8-f93bf14fea73 file = illumos-utils/src/zone.rs:354
10052024-08-28T04:04:37.706ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_4b839573-b270-4df0-b615-f92964eededd file = illumos-utils/src/zone.rs:354
10062024-08-28T04:04:37.710ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_oximeter_21378de4-15c3-48e2-b919-b1cc2363c4a6 file = illumos-utils/src/zone.rs:354
10072024-08-28T04:04:37.714ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_d073ae9b-afee-401e-bafb-45a05ad31f0d file = illumos-utils/src/zone.rs:354
10082024-08-28T04:04:37.719ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_ff9eaaaf-6eec-47be-bc80-11d83c51090f file = illumos-utils/src/zone.rs:354
10092024-08-28T04:04:37.723ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_7e16ed01-11ff-45eb-8869-6ed06d9b0e70 file = illumos-utils/src/zone.rs:354
10102024-08-28T04:04:37.728ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_a8d89411-e3d4-4290-8995-191bd8cdd564 file = illumos-utils/src/zone.rs:354
10112024-08-28T04:04:37.734ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_clickhouse_f62eaa35-1217-424b-a535-2aa55c06fbad file = illumos-utils/src/zone.rs:316
10122024-08-28T04:04:37.739ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_1d429812-8d5a-47a0-98f7-1f1f5ef6e285 file = illumos-utils/src/zone.rs:316
10132024-08-28T04:04:37.744ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_83266762-372c-41cd-a9dd-eee1b2c38bb6 file = illumos-utils/src/zone.rs:316
10142024-08-28T04:04:37.749ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_d56672e9-8c1d-4f9a-a4b1-49fa3e1cfbe0 file = illumos-utils/src/zone.rs:316
10152024-08-28T04:04:37.756ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_7f6ebc53-fc17-488d-9b99-8dc983f4efec file = illumos-utils/src/zone.rs:316
10162024-08-28T04:04:37.762ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_24b1ac7b-c1d1-440a-a4e4-220c7ce34410 file = illumos-utils/src/zone.rs:316
10172024-08-28T04:04:37.769ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_clickhouse_f62eaa35-1217-424b-a535-2aa55c06fbad file = illumos-utils/src/zone.rs:354
10182024-08-28T04:04:37.776ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_1d429812-8d5a-47a0-98f7-1f1f5ef6e285 file = illumos-utils/src/zone.rs:354
10192024-08-28T04:04:37.781ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_83266762-372c-41cd-a9dd-eee1b2c38bb6 file = illumos-utils/src/zone.rs:354
10202024-08-28T04:04:37.787ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_d56672e9-8c1d-4f9a-a4b1-49fa3e1cfbe0 file = illumos-utils/src/zone.rs:354
10212024-08-28T04:04:37.793ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_7f6ebc53-fc17-488d-9b99-8dc983f4efec file = illumos-utils/src/zone.rs:354
10222024-08-28T04:04:37.798ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_24b1ac7b-c1d1-440a-a4e4-220c7ce34410 file = illumos-utils/src/zone.rs:354
10232024-08-28T04:04:37.842ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
10242024-08-28T04:04:37.885ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
10252024-08-28T04:04:37.942ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
10262024-08-28T04:04:38.000ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
10272024-08-28T04:04:38.060ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
10282024-08-28T04:04:38.061ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
10292024-08-28T04:04:38.061ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
10302024-08-28T04:04:38.194ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte1", ip: 172.30.2.7, mac: MacAddr6([168, 64, 37, 255, 210, 255]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
10312024-08-28T04:04:38.194ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1235 zone_type = nexus
10322024-08-28T04:04:38.194ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
10332024-08-28T04:04:38.195ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
10342024-08-28T04:04:38.249ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte2", ip: 172.30.1.5, mac: MacAddr6([168, 64, 37, 255, 246, 113]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.1.1 } }) }
10352024-08-28T04:04:38.249ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1235 zone_type = external_dns
10362024-08-28T04:04:38.251ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
10372024-08-28T04:04:38.251ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
10382024-08-28T04:04:38.304ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte3", ip: 172.30.1.6, mac: MacAddr6([168, 64, 37, 255, 165, 179]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.1.1 } }) }
10392024-08-28T04:04:38.304ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1235 zone_type = external_dns
10402024-08-28T04:04:38.305ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
10412024-08-28T04:04:38.305ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
10422024-08-28T04:04:38.362ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte4", ip: 172.30.2.6, mac: MacAddr6([168, 64, 37, 255, 208, 245]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
10432024-08-28T04:04:38.362ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1235 zone_type = nexus
10442024-08-28T04:04:38.366ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
10452024-08-28T04:04:38.366ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
10462024-08-28T04:04:38.421ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte5", ip: 172.30.2.5, mac: MacAddr6([168, 64, 37, 255, 207, 220]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
10472024-08-28T04:04:38.421ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1235 zone_type = nexus
10482024-08-28T04:04:38.422ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.168 file = clients/dpd-client/src/lib.rs:83
10492024-08-28T04:04:38.422ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.168 file = clients/dpd-client/src/lib.rs:90
10502024-08-28T04:04:38.422ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.164 file = clients/dpd-client/src/lib.rs:83
10512024-08-28T04:04:38.422ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.164 file = clients/dpd-client/src/lib.rs:90
10522024-08-28T04:04:38.422ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.165 file = clients/dpd-client/src/lib.rs:83
10532024-08-28T04:04:38.422ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.165 file = clients/dpd-client/src/lib.rs:90
10542024-08-28T04:04:38.422ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.167 file = clients/dpd-client/src/lib.rs:83
10552024-08-28T04:04:38.422ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.167 file = clients/dpd-client/src/lib.rs:90
10562024-08-28T04:04:38.430ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.168 file = clients/dpd-client/src/lib.rs:118
10572024-08-28T04:04:38.431ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
10582024-08-28T04:04:38.482ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.164 file = clients/dpd-client/src/lib.rs:118
10592024-08-28T04:04:38.487ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.165 file = clients/dpd-client/src/lib.rs:118
10602024-08-28T04:04:38.490ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
10612024-08-28T04:04:38.490ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
10622024-08-28T04:04:38.490ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.167 file = clients/dpd-client/src/lib.rs:118
10632024-08-28T04:04:38.493ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.166 file = clients/dpd-client/src/lib.rs:83
10642024-08-28T04:04:38.493ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.166 file = clients/dpd-client/src/lib.rs:90
10652024-08-28T04:04:38.493ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
10662024-08-28T04:04:38.573ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:48333
10672024-08-28T04:04:38.576ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2779 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:48333 req_id = 54bb0f0f-0583-4587-bb9c-dfc5a6ffa2ce response_code = 200 uri = /baseboard
10682024-08-28T04:04:38.770ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.166 file = clients/dpd-client/src/lib.rs:118
10692024-08-28T04:04:38.775ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_bda2ed8e-2ea5-41eb-824b-d14527b76b8b file = illumos-utils/src/zone.rs:316
10702024-08-28T04:04:38.796ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
10712024-08-28T04:04:38.797ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_external_dns_4eb364e4-14a2-46c4-b173-eb2890920e8f file = illumos-utils/src/zone.rs:316
10722024-08-28T04:04:38.815ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_external_dns_94b35d7a-af71-4078-a57c-874ff9a27068 file = illumos-utils/src/zone.rs:316
10732024-08-28T04:04:38.827ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_1c6ee351-cab6-4c23-8658-c31bac40b40a file = illumos-utils/src/zone.rs:316
10742024-08-28T04:04:39.245ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_bda2ed8e-2ea5-41eb-824b-d14527b76b8b file = illumos-utils/src/zone.rs:354
10752024-08-28T04:04:39.293ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_external_dns_4eb364e4-14a2-46c4-b173-eb2890920e8f file = illumos-utils/src/zone.rs:354
10762024-08-28T04:04:39.331ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_external_dns_94b35d7a-af71-4078-a57c-874ff9a27068 file = illumos-utils/src/zone.rs:354
10772024-08-28T04:04:39.339ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_1c6ee351-cab6-4c23-8658-c31bac40b40a file = illumos-utils/src/zone.rs:354
10782024-08-28T04:04:39.347ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_4c8a8ded-ced6-466f-8626-7fd39c37d020 file = illumos-utils/src/zone.rs:316
10792024-08-28T04:04:39.916ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_4c8a8ded-ced6-466f-8626-7fd39c37d020 file = illumos-utils/src/zone.rs:354
10802024-08-28T04:04:44.972ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_d56672e9-8c1d-4f9a-a4b1-49fa3e1cfbe0: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService21'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::11'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::11'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10812024-08-28T04:04:44.974ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_d56672e9-8c1d-4f9a-a4b1-49fa3e1cfbe0 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_d56672e9-8c1d-4f9a-a4b1-49fa3e1cfbe0
10822024-08-28T04:04:45.172ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_a8d89411-e3d4-4290-8995-191bd8cdd564: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService17'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::19'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_ed7027bd-3cf9-4764-bc15-404c842ab02a/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::19'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='0b07db32-10ad-45b8-a41c-f8317af72ed6'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10832024-08-28T04:04:45.176ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_24b1ac7b-c1d1-440a-a4e4-220c7ce34410: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService23'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::13'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_0f906b25-a03c-4334-807e-07a343a51361/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::13'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='842cd0b5-cf92-40a7-8c48-07c8a98ca748'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10842024-08-28T04:04:45.176ZINFOSledAgent (ServiceManager): Booting oxz_crucible_a8d89411-e3d4-4290-8995-191bd8cdd564 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_a8d89411-e3d4-4290-8995-191bd8cdd564
10852024-08-28T04:04:45.187ZINFOSledAgent (ServiceManager): Booting oxz_crucible_24b1ac7b-c1d1-440a-a4e4-220c7ce34410 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_24b1ac7b-c1d1-440a-a4e4-220c7ce34410
10862024-08-28T04:04:45.199ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_d6ce85cc-4f25-4b5e-b0c0-a434368f63d4: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService10'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::14'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_d6170dc9-a1d4-44a9-ba24-00c07a403b1e/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::14'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='33918000-c90c-4edf-b4a9-01e740482253'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10872024-08-28T04:04:45.204ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_d4f01887-9496-477a-9ed8-f93bf14fea73: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService11'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::1a'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_e3fa46c7-4f3a-45f7-afd8-b7799f191587/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::1a'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='65e4134d-f30c-4c6f-af76-48c395f679d4'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10882024-08-28T04:04:45.205ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_4b839573-b270-4df0-b615-f92964eededd: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService12'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::12'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_57750eb9-c738-4eb8-89f2-c5eab164d34c/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::12'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='6bc59c00-f27b-46f1-a608-1ee55ab0982f'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10892024-08-28T04:04:45.222ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_d073ae9b-afee-401e-bafb-45a05ad31f0d: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService14'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::15'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_8955d8a9-1b85-47ea-bd5c-275ef6dc0b1a/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::15'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='0d9f5a60-14cb-46f7-aed9-2ded9451deda'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10902024-08-28T04:04:45.229ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_ff9eaaaf-6eec-47be-bc80-11d83c51090f: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService15'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::16'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_fb7c10cb-4fd0-49dd-8c21-f83feafbd7dc/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::16'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='55c11dfe-5b52-4cc5-bd13-df3f006fc465'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10912024-08-28T04:04:45.239ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_7f6ebc53-fc17-488d-9b99-8dc983f4efec: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService22'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::18'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_0913dfa8-99f2-4518-8d37-827ea32c30ba/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::18'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='cecd2675-0bc9-47f8-a0ff-b7eda529e2ec'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10922024-08-28T04:04:45.247ZINFOSledAgent (ServiceManager): Booting oxz_crucible_d6ce85cc-4f25-4b5e-b0c0-a434368f63d4 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_d6ce85cc-4f25-4b5e-b0c0-a434368f63d4
10932024-08-28T04:04:45.252ZINFOSledAgent (ServiceManager): Booting oxz_crucible_d4f01887-9496-477a-9ed8-f93bf14fea73 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_d4f01887-9496-477a-9ed8-f93bf14fea73
10942024-08-28T04:04:45.315ZINFOSledAgent (ServiceManager): Booting oxz_crucible_4b839573-b270-4df0-b615-f92964eededd zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_4b839573-b270-4df0-b615-f92964eededd
10952024-08-28T04:04:45.326ZINFOSledAgent (ServiceManager): Booting oxz_crucible_d073ae9b-afee-401e-bafb-45a05ad31f0d zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_d073ae9b-afee-401e-bafb-45a05ad31f0d
10962024-08-28T04:04:45.342ZINFOSledAgent (ServiceManager): Booting oxz_crucible_ff9eaaaf-6eec-47be-bc80-11d83c51090f zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_ff9eaaaf-6eec-47be-bc80-11d83c51090f
10972024-08-28T04:04:45.377ZINFOSledAgent (ServiceManager): Booting oxz_crucible_7f6ebc53-fc17-488d-9b99-8dc983f4efec zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_7f6ebc53-fc17-488d-9b99-8dc983f4efec
10982024-08-28T04:04:58.150ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d6ce85cc-4f25-4b5e-b0c0-a434368f63d4") failed: Property not found. retry in 37.277797ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d6ce85cc-4f25-4b5e-b0c0-a434368f63d4
10992024-08-28T04:04:58.245ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d4f01887-9496-477a-9ed8-f93bf14fea73") failed: Property not found. retry in 30.346781ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d4f01887-9496-477a-9ed8-f93bf14fea73
11002024-08-28T04:04:58.379ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b839573-b270-4df0-b615-f92964eededd") failed: Property not found. retry in 50.881192ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b839573-b270-4df0-b615-f92964eededd
11012024-08-28T04:04:58.429ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d073ae9b-afee-401e-bafb-45a05ad31f0d") failed: Property not found. retry in 34.260934ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d073ae9b-afee-401e-bafb-45a05ad31f0d
11022024-08-28T04:04:58.612ZINFOSledAgent (ServiceManager): Profile for oxz_external_dns_4eb364e4-14a2-46c4-b173-eb2890920e8f: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService25'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::8'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte2'/> <propval type="astring" name="gateway" value='172.30.1.1'/> <propval type="astring" name="ip" value='172.30.1.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/external_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::8]:5353'/> <propval type="astring" name="dns_address" value='172.30.1.5:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11032024-08-28T04:04:58.710ZINFOSledAgent (ServiceManager): Profile for oxz_external_dns_94b35d7a-af71-4078-a57c-874ff9a27068: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService26'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::9'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte3'/> <propval type="astring" name="gateway" value='172.30.1.1'/> <propval type="astring" name="ip" value='172.30.1.6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/external_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::9]:5353'/> <propval type="astring" name="dns_address" value='172.30.1.6:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11042024-08-28T04:04:58.786ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ff9eaaaf-6eec-47be-bc80-11d83c51090f") failed: Property not found. retry in 59.264504ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ff9eaaaf-6eec-47be-bc80-11d83c51090f
11052024-08-28T04:04:58.832ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_7e16ed01-11ff-45eb-8869-6ed06d9b0e70: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService16'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::17'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_ba3c519e-f98a-4ee2-a345-a21845f5f7a4/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::17'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='33c149f9-7eb6-4ddb-997f-8ba0e90a6683'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11062024-08-28T04:04:58.898ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a8d89411-e3d4-4290-8995-191bd8cdd564") failed: Property not found. retry in 64.15406ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a8d89411-e3d4-4290-8995-191bd8cdd564
11072024-08-28T04:04:58.901ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_1d429812-8d5a-47a0-98f7-1f1f5ef6e285: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService19'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::10'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::10'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11082024-08-28T04:04:58.910ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_83266762-372c-41cd-a9dd-eee1b2c38bb6: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService20'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::f'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::f'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11092024-08-28T04:04:58.926ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d56672e9-8c1d-4f9a-a4b1-49fa3e1cfbe0") failed: Property not found. retry in 25.687984ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d56672e9-8c1d-4f9a-a4b1-49fa3e1cfbe0
11102024-08-28T04:04:58.939ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7f6ebc53-fc17-488d-9b99-8dc983f4efec") failed: Property not found. retry in 54.523449ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7f6ebc53-fc17-488d-9b99-8dc983f4efec
11112024-08-28T04:04:58.980ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_24b1ac7b-c1d1-440a-a4e4-220c7ce34410") failed: Property not found. retry in 69.506516ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_24b1ac7b-c1d1-440a-a4e4-220c7ce34410
11122024-08-28T04:04:58.994ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d6ce85cc-4f25-4b5e-b0c0-a434368f63d4") failed: Property not found. retry in 127.804776ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d6ce85cc-4f25-4b5e-b0c0-a434368f63d4
11132024-08-28T04:04:59.046ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d4f01887-9496-477a-9ed8-f93bf14fea73") failed: Property not found. retry in 62.385305ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d4f01887-9496-477a-9ed8-f93bf14fea73
11142024-08-28T04:04:59.085ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b839573-b270-4df0-b615-f92964eededd") failed: Property not found. retry in 60.078598ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b839573-b270-4df0-b615-f92964eededd
11152024-08-28T04:04:59.105ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d073ae9b-afee-401e-bafb-45a05ad31f0d") failed: Property not found. retry in 105.230427ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d073ae9b-afee-401e-bafb-45a05ad31f0d
11162024-08-28T04:04:59.106ZINFOSledAgent (ServiceManager): Booting oxz_external_dns_4eb364e4-14a2-46c4-b173-eb2890920e8f zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_external_dns_4eb364e4-14a2-46c4-b173-eb2890920e8f
11172024-08-28T04:04:59.124ZINFOSledAgent (ServiceManager): Booting oxz_external_dns_94b35d7a-af71-4078-a57c-874ff9a27068 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_external_dns_94b35d7a-af71-4078-a57c-874ff9a27068
11182024-08-28T04:04:59.163ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ff9eaaaf-6eec-47be-bc80-11d83c51090f") failed: Property not found. retry in 107.094214ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ff9eaaaf-6eec-47be-bc80-11d83c51090f
11192024-08-28T04:04:59.173ZINFOSledAgent (ServiceManager): Booting oxz_crucible_7e16ed01-11ff-45eb-8869-6ed06d9b0e70 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_7e16ed01-11ff-45eb-8869-6ed06d9b0e70
11202024-08-28T04:04:59.213ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a8d89411-e3d4-4290-8995-191bd8cdd564") failed: Property not found. retry in 114.659985ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a8d89411-e3d4-4290-8995-191bd8cdd564
11212024-08-28T04:04:59.213ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_1d429812-8d5a-47a0-98f7-1f1f5ef6e285 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_1d429812-8d5a-47a0-98f7-1f1f5ef6e285
11222024-08-28T04:04:59.239ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_83266762-372c-41cd-a9dd-eee1b2c38bb6 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_83266762-372c-41cd-a9dd-eee1b2c38bb6
11232024-08-28T04:04:59.273ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d56672e9-8c1d-4f9a-a4b1-49fa3e1cfbe0") failed: Property not found. retry in 130.631797ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d56672e9-8c1d-4f9a-a4b1-49fa3e1cfbe0
11242024-08-28T04:04:59.291ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7f6ebc53-fc17-488d-9b99-8dc983f4efec") failed: Property not found. retry in 50.856227ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7f6ebc53-fc17-488d-9b99-8dc983f4efec
11252024-08-28T04:04:59.310ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_24b1ac7b-c1d1-440a-a4e4-220c7ce34410") failed: Property not found. retry in 58.964783ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_24b1ac7b-c1d1-440a-a4e4-220c7ce34410
11262024-08-28T04:04:59.344ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d6ce85cc-4f25-4b5e-b0c0-a434368f63d4") failed: Property not found. retry in 208.074755ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d6ce85cc-4f25-4b5e-b0c0-a434368f63d4
11272024-08-28T04:04:59.374ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d4f01887-9496-477a-9ed8-f93bf14fea73") failed: Property not found. retry in 244.696802ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d4f01887-9496-477a-9ed8-f93bf14fea73
11282024-08-28T04:04:59.397ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 167.386024575s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f
11292024-08-28T04:04:59.403ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b839573-b270-4df0-b615-f92964eededd") failed: Property not found. retry in 229.649472ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b839573-b270-4df0-b615-f92964eededd
11302024-08-28T04:04:59.438ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d073ae9b-afee-401e-bafb-45a05ad31f0d") failed: Property not found. retry in 285.853889ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d073ae9b-afee-401e-bafb-45a05ad31f0d
11312024-08-28T04:04:59.464ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_4eb364e4-14a2-46c4-b173-eb2890920e8f") failed: Property not found. retry in 31.029641ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_4eb364e4-14a2-46c4-b173-eb2890920e8f
11322024-08-28T04:04:59.537ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_94b35d7a-af71-4078-a57c-874ff9a27068") failed: Property not found. retry in 48.345078ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_94b35d7a-af71-4078-a57c-874ff9a27068
11332024-08-28T04:04:59.566ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ff9eaaaf-6eec-47be-bc80-11d83c51090f") failed: Property not found. retry in 110.899877ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ff9eaaaf-6eec-47be-bc80-11d83c51090f
11342024-08-28T04:04:59.593ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7e16ed01-11ff-45eb-8869-6ed06d9b0e70") failed: Property not found. retry in 70.876356ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7e16ed01-11ff-45eb-8869-6ed06d9b0e70
11352024-08-28T04:04:59.616ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a8d89411-e3d4-4290-8995-191bd8cdd564") failed: Property not found. retry in 278.769832ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a8d89411-e3d4-4290-8995-191bd8cdd564
11362024-08-28T04:04:59.630ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1d429812-8d5a-47a0-98f7-1f1f5ef6e285") failed: Property not found. retry in 50.531874ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1d429812-8d5a-47a0-98f7-1f1f5ef6e285
11372024-08-28T04:04:59.640ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_83266762-372c-41cd-a9dd-eee1b2c38bb6") failed: Property not found. retry in 47.323662ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_83266762-372c-41cd-a9dd-eee1b2c38bb6
11382024-08-28T04:04:59.654ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d56672e9-8c1d-4f9a-a4b1-49fa3e1cfbe0") failed: Property not found. retry in 187.310773ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d56672e9-8c1d-4f9a-a4b1-49fa3e1cfbe0
11392024-08-28T04:04:59.670ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7f6ebc53-fc17-488d-9b99-8dc983f4efec") failed: Property not found. retry in 275.058965ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7f6ebc53-fc17-488d-9b99-8dc983f4efec
11402024-08-28T04:04:59.684ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_24b1ac7b-c1d1-440a-a4e4-220c7ce34410") failed: Property not found. retry in 163.464553ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_24b1ac7b-c1d1-440a-a4e4-220c7ce34410
11412024-08-28T04:04:59.702ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d6ce85cc-4f25-4b5e-b0c0-a434368f63d4") failed: Property not found. retry in 347.798877ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d6ce85cc-4f25-4b5e-b0c0-a434368f63d4
11422024-08-28T04:04:59.716ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d4f01887-9496-477a-9ed8-f93bf14fea73") failed: Property not found. retry in 303.903404ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d4f01887-9496-477a-9ed8-f93bf14fea73
11432024-08-28T04:04:59.734ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b839573-b270-4df0-b615-f92964eededd") failed: Property not found. retry in 242.964155ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b839573-b270-4df0-b615-f92964eededd
11442024-08-28T04:04:59.746ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d073ae9b-afee-401e-bafb-45a05ad31f0d") failed: Property not found. retry in 528.330049ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d073ae9b-afee-401e-bafb-45a05ad31f0d
11452024-08-28T04:04:59.761ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_4eb364e4-14a2-46c4-b173-eb2890920e8f") failed: Property not found. retry in 96.257287ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_4eb364e4-14a2-46c4-b173-eb2890920e8f
11462024-08-28T04:04:59.772ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_94b35d7a-af71-4078-a57c-874ff9a27068") failed: Property not found. retry in 112.008157ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_94b35d7a-af71-4078-a57c-874ff9a27068
11472024-08-28T04:04:59.786ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ff9eaaaf-6eec-47be-bc80-11d83c51090f") failed: Property not found. retry in 386.091187ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ff9eaaaf-6eec-47be-bc80-11d83c51090f
11482024-08-28T04:04:59.811ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7e16ed01-11ff-45eb-8869-6ed06d9b0e70") failed: Property not found. retry in 71.071529ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7e16ed01-11ff-45eb-8869-6ed06d9b0e70
11492024-08-28T04:04:59.821ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1d429812-8d5a-47a0-98f7-1f1f5ef6e285") failed: Property not found. retry in 77.113733ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1d429812-8d5a-47a0-98f7-1f1f5ef6e285
11502024-08-28T04:04:59.844ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_83266762-372c-41cd-a9dd-eee1b2c38bb6") failed: Property not found. retry in 86.488888ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_83266762-372c-41cd-a9dd-eee1b2c38bb6
11512024-08-28T04:04:59.860ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d56672e9-8c1d-4f9a-a4b1-49fa3e1cfbe0") failed: Property not found. retry in 580.346939ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d56672e9-8c1d-4f9a-a4b1-49fa3e1cfbe0
11522024-08-28T04:04:59.883ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_24b1ac7b-c1d1-440a-a4e4-220c7ce34410") failed: Property not found. retry in 509.641856ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_24b1ac7b-c1d1-440a-a4e4-220c7ce34410
11532024-08-28T04:04:59.894ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_4eb364e4-14a2-46c4-b173-eb2890920e8f") failed: Property not found. retry in 236.571183ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_4eb364e4-14a2-46c4-b173-eb2890920e8f
11542024-08-28T04:04:59.903ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_94b35d7a-af71-4078-a57c-874ff9a27068") failed: Property not found. retry in 242.463157ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_94b35d7a-af71-4078-a57c-874ff9a27068
11552024-08-28T04:04:59.916ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7e16ed01-11ff-45eb-8869-6ed06d9b0e70") failed: Property not found. retry in 196.454813ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7e16ed01-11ff-45eb-8869-6ed06d9b0e70
11562024-08-28T04:04:59.925ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a8d89411-e3d4-4290-8995-191bd8cdd564") failed: Property not found. retry in 582.938881ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a8d89411-e3d4-4290-8995-191bd8cdd564
11572024-08-28T04:04:59.951ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1d429812-8d5a-47a0-98f7-1f1f5ef6e285") failed: Property not found. retry in 271.297047ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1d429812-8d5a-47a0-98f7-1f1f5ef6e285
11582024-08-28T04:04:59.967ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_83266762-372c-41cd-a9dd-eee1b2c38bb6") failed: Property not found. retry in 210.386519ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_83266762-372c-41cd-a9dd-eee1b2c38bb6
11592024-08-28T04:04:59.995ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7f6ebc53-fc17-488d-9b99-8dc983f4efec") failed: Property not found. retry in 226.764511ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7f6ebc53-fc17-488d-9b99-8dc983f4efec
11602024-08-28T04:05:00.007ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b839573-b270-4df0-b615-f92964eededd") failed: Property not found. retry in 687.660084ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b839573-b270-4df0-b615-f92964eededd
11612024-08-28T04:05:00.036ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d4f01887-9496-477a-9ed8-f93bf14fea73") failed: Property not found. retry in 486.606595ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d4f01887-9496-477a-9ed8-f93bf14fea73
11622024-08-28T04:05:00.061ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d6ce85cc-4f25-4b5e-b0c0-a434368f63d4") failed: Property not found. retry in 545.917789ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d6ce85cc-4f25-4b5e-b0c0-a434368f63d4
11632024-08-28T04:05:00.135ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7e16ed01-11ff-45eb-8869-6ed06d9b0e70") failed: Property not found. retry in 353.616882ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7e16ed01-11ff-45eb-8869-6ed06d9b0e70
11642024-08-28T04:05:00.173ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_4eb364e4-14a2-46c4-b173-eb2890920e8f") failed: Property not found. retry in 538.797985ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_4eb364e4-14a2-46c4-b173-eb2890920e8f
11652024-08-28T04:05:00.179ZINFOSledAgent (ServiceManager): Profile for oxz_oximeter_21378de4-15c3-48e2-b919-b1cc2363c4a6: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService13'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::d'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/oximeter"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="id" value='21378de4-15c3-48e2-b919-b1cc2363c4a6'/> <propval type="astring" name="address" value='[fd00:1122:3344:101::d]:12223'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11662024-08-28T04:05:00.192ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_94b35d7a-af71-4078-a57c-874ff9a27068") failed: Property not found. retry in 204.845484ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_94b35d7a-af71-4078-a57c-874ff9a27068
11672024-08-28T04:05:00.209ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ff9eaaaf-6eec-47be-bc80-11d83c51090f") failed: Property not found. retry in 593.960003ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ff9eaaaf-6eec-47be-bc80-11d83c51090f
11682024-08-28T04:05:00.210ZINFOSledAgent (ServiceManager): Booting oxz_oximeter_21378de4-15c3-48e2-b919-b1cc2363c4a6 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_oximeter_21378de4-15c3-48e2-b919-b1cc2363c4a6
11692024-08-28T04:05:00.243ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1d429812-8d5a-47a0-98f7-1f1f5ef6e285") failed: Property not found. retry in 576.635662ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1d429812-8d5a-47a0-98f7-1f1f5ef6e285
11702024-08-28T04:05:00.258ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_83266762-372c-41cd-a9dd-eee1b2c38bb6") failed: Property not found. retry in 507.483996ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_83266762-372c-41cd-a9dd-eee1b2c38bb6
11712024-08-28T04:05:00.287ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7f6ebc53-fc17-488d-9b99-8dc983f4efec") failed: Property not found. retry in 1.182192872s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7f6ebc53-fc17-488d-9b99-8dc983f4efec
11722024-08-28T04:05:00.303ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d073ae9b-afee-401e-bafb-45a05ad31f0d") failed: Property not found. retry in 1.175454149s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d073ae9b-afee-401e-bafb-45a05ad31f0d
11732024-08-28T04:05:00.440ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_21378de4-15c3-48e2-b919-b1cc2363c4a6") failed: Property not found. retry in 73.69968ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_21378de4-15c3-48e2-b919-b1cc2363c4a6
11742024-08-28T04:05:00.461ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_94b35d7a-af71-4078-a57c-874ff9a27068") failed: Property not found. retry in 526.073283ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_94b35d7a-af71-4078-a57c-874ff9a27068
11752024-08-28T04:05:00.482ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_24b1ac7b-c1d1-440a-a4e4-220c7ce34410") failed: Property not found. retry in 745.105903ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_24b1ac7b-c1d1-440a-a4e4-220c7ce34410
11762024-08-28T04:05:00.493ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d56672e9-8c1d-4f9a-a4b1-49fa3e1cfbe0") failed: Property not found. retry in 892.729797ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d56672e9-8c1d-4f9a-a4b1-49fa3e1cfbe0
11772024-08-28T04:05:00.505ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7e16ed01-11ff-45eb-8869-6ed06d9b0e70") failed: Property not found. retry in 1.123597473s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7e16ed01-11ff-45eb-8869-6ed06d9b0e70
11782024-08-28T04:05:00.517ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a8d89411-e3d4-4290-8995-191bd8cdd564") failed: Property not found. retry in 598.433604ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a8d89411-e3d4-4290-8995-191bd8cdd564
11792024-08-28T04:05:00.525ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_21378de4-15c3-48e2-b919-b1cc2363c4a6") failed: Property not found. retry in 67.632371ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_21378de4-15c3-48e2-b919-b1cc2363c4a6
11802024-08-28T04:05:00.534ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d4f01887-9496-477a-9ed8-f93bf14fea73") failed: Property not found. retry in 1.178713812s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d4f01887-9496-477a-9ed8-f93bf14fea73
11812024-08-28T04:05:00.605ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_21378de4-15c3-48e2-b919-b1cc2363c4a6") failed: Property not found. retry in 148.165704ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_21378de4-15c3-48e2-b919-b1cc2363c4a6
11822024-08-28T04:05:00.617ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d6ce85cc-4f25-4b5e-b0c0-a434368f63d4") failed: Property not found. retry in 1.222470179s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d6ce85cc-4f25-4b5e-b0c0-a434368f63d4
11832024-08-28T04:05:00.706ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b839573-b270-4df0-b615-f92964eededd") failed: Property not found. retry in 1.186410259s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b839573-b270-4df0-b615-f92964eededd
11842024-08-28T04:05:00.722ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_4eb364e4-14a2-46c4-b173-eb2890920e8f") failed: Property not found. retry in 607.456713ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_4eb364e4-14a2-46c4-b173-eb2890920e8f
11852024-08-28T04:05:00.763ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_21378de4-15c3-48e2-b919-b1cc2363c4a6") failed: Property not found. retry in 303.185512ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_21378de4-15c3-48e2-b919-b1cc2363c4a6
11862024-08-28T04:05:00.776ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_83266762-372c-41cd-a9dd-eee1b2c38bb6") failed: Property not found. retry in 448.579943ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_83266762-372c-41cd-a9dd-eee1b2c38bb6
11872024-08-28T04:05:00.813ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ff9eaaaf-6eec-47be-bc80-11d83c51090f") failed: Property not found. retry in 722.548909ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ff9eaaaf-6eec-47be-bc80-11d83c51090f
11882024-08-28T04:05:00.830ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1d429812-8d5a-47a0-98f7-1f1f5ef6e285") failed: Property not found. retry in 819.353057ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1d429812-8d5a-47a0-98f7-1f1f5ef6e285
11892024-08-28T04:05:00.997ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_94b35d7a-af71-4078-a57c-874ff9a27068") failed: Property not found. retry in 1.021811076s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_94b35d7a-af71-4078-a57c-874ff9a27068
11902024-08-28T04:05:01.079ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_21378de4-15c3-48e2-b919-b1cc2363c4a6") failed: Property not found. retry in 915.444174ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_21378de4-15c3-48e2-b919-b1cc2363c4a6
11912024-08-28T04:05:01.125ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a8d89411-e3d4-4290-8995-191bd8cdd564") failed: Property not found. retry in 1.02325409s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a8d89411-e3d4-4290-8995-191bd8cdd564
11922024-08-28T04:05:01.236ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_83266762-372c-41cd-a9dd-eee1b2c38bb6") failed: Property not found. retry in 530.089569ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_83266762-372c-41cd-a9dd-eee1b2c38bb6
11932024-08-28T04:05:01.245ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_24b1ac7b-c1d1-440a-a4e4-220c7ce34410") failed: Property not found. retry in 1.175364758s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_24b1ac7b-c1d1-440a-a4e4-220c7ce34410
11942024-08-28T04:05:01.338ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_4eb364e4-14a2-46c4-b173-eb2890920e8f") failed: Property not found. retry in 993.421417ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_4eb364e4-14a2-46c4-b173-eb2890920e8f
11952024-08-28T04:05:01.404ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d56672e9-8c1d-4f9a-a4b1-49fa3e1cfbe0") failed: Property not found. retry in 618.18455ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d56672e9-8c1d-4f9a-a4b1-49fa3e1cfbe0
11962024-08-28T04:05:01.480ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7f6ebc53-fc17-488d-9b99-8dc983f4efec") failed: Property not found. retry in 529.560153ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7f6ebc53-fc17-488d-9b99-8dc983f4efec
11972024-08-28T04:05:01.489ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d073ae9b-afee-401e-bafb-45a05ad31f0d") failed: Property not found. retry in 727.228989ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d073ae9b-afee-401e-bafb-45a05ad31f0d
11982024-08-28T04:05:01.546ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ff9eaaaf-6eec-47be-bc80-11d83c51090f") failed: Property not found. retry in 692.607887ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ff9eaaaf-6eec-47be-bc80-11d83c51090f
11992024-08-28T04:05:01.640ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7e16ed01-11ff-45eb-8869-6ed06d9b0e70") failed: Property not found. retry in 1.211573422s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7e16ed01-11ff-45eb-8869-6ed06d9b0e70
12002024-08-28T04:05:01.659ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1d429812-8d5a-47a0-98f7-1f1f5ef6e285") failed: Property not found. retry in 689.881086ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1d429812-8d5a-47a0-98f7-1f1f5ef6e285
12012024-08-28T04:05:01.723ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d4f01887-9496-477a-9ed8-f93bf14fea73") failed: Property not found. retry in 1.227394442s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d4f01887-9496-477a-9ed8-f93bf14fea73
12022024-08-28T04:05:01.778ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_83266762-372c-41cd-a9dd-eee1b2c38bb6") failed: Property not found. retry in 898.57284ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_83266762-372c-41cd-a9dd-eee1b2c38bb6
12032024-08-28T04:05:01.850ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d6ce85cc-4f25-4b5e-b0c0-a434368f63d4") failed: Property not found. retry in 1.048188321s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d6ce85cc-4f25-4b5e-b0c0-a434368f63d4
12042024-08-28T04:05:01.903ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b839573-b270-4df0-b615-f92964eededd") failed: Property not found. retry in 1.182659298s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b839573-b270-4df0-b615-f92964eededd
12052024-08-28T04:05:01.979ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_1c6ee351-cab6-4c23-8658-c31bac40b40a: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService27'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::b'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte4'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
12062024-08-28T04:05:01.980ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_bda2ed8e-2ea5-41eb-824b-d14527b76b8b: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService24'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::c'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte1'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.7'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
12072024-08-28T04:05:01.980ZINFOSledAgent (ServiceManager): Booting oxz_nexus_1c6ee351-cab6-4c23-8658-c31bac40b40a zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_1c6ee351-cab6-4c23-8658-c31bac40b40a
12082024-08-28T04:05:01.986ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_4c8a8ded-ced6-466f-8626-7fd39c37d020: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService28'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::a'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte5'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
12092024-08-28T04:05:01.986ZINFOSledAgent (ServiceManager): Booting oxz_nexus_bda2ed8e-2ea5-41eb-824b-d14527b76b8b zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_bda2ed8e-2ea5-41eb-824b-d14527b76b8b
12102024-08-28T04:05:01.992ZINFOSledAgent (ServiceManager): Booting oxz_nexus_4c8a8ded-ced6-466f-8626-7fd39c37d020 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_4c8a8ded-ced6-466f-8626-7fd39c37d020
12112024-08-28T04:05:02.009ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_21378de4-15c3-48e2-b919-b1cc2363c4a6") failed: Property not found. retry in 1.017012355s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_21378de4-15c3-48e2-b919-b1cc2363c4a6
12122024-08-28T04:05:02.022ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7f6ebc53-fc17-488d-9b99-8dc983f4efec") failed: Property not found. retry in 825.506698ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7f6ebc53-fc17-488d-9b99-8dc983f4efec
12132024-08-28T04:05:02.033ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_94b35d7a-af71-4078-a57c-874ff9a27068") failed: Property not found. retry in 638.674904ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_94b35d7a-af71-4078-a57c-874ff9a27068
12142024-08-28T04:05:02.045ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d56672e9-8c1d-4f9a-a4b1-49fa3e1cfbe0") failed: Property not found. retry in 1.136482796s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d56672e9-8c1d-4f9a-a4b1-49fa3e1cfbe0
12152024-08-28T04:05:02.161ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a8d89411-e3d4-4290-8995-191bd8cdd564") failed: Property not found. retry in 1.401831836s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a8d89411-e3d4-4290-8995-191bd8cdd564
12162024-08-28T04:05:02.190ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1c6ee351-cab6-4c23-8658-c31bac40b40a") failed: Property not found. retry in 50.60332ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1c6ee351-cab6-4c23-8658-c31bac40b40a
12172024-08-28T04:05:02.204ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4c8a8ded-ced6-466f-8626-7fd39c37d020") failed: Property not found. retry in 57.76446ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4c8a8ded-ced6-466f-8626-7fd39c37d020
12182024-08-28T04:05:02.212ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_bda2ed8e-2ea5-41eb-824b-d14527b76b8b") failed: Property not found. retry in 43.336059ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_bda2ed8e-2ea5-41eb-824b-d14527b76b8b
12192024-08-28T04:05:02.228ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d073ae9b-afee-401e-bafb-45a05ad31f0d") failed: Property not found. retry in 1.039726508s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d073ae9b-afee-401e-bafb-45a05ad31f0d
12202024-08-28T04:05:02.250ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ff9eaaaf-6eec-47be-bc80-11d83c51090f") failed: Property not found. retry in 872.956002ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ff9eaaaf-6eec-47be-bc80-11d83c51090f
12212024-08-28T04:05:02.262ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1c6ee351-cab6-4c23-8658-c31bac40b40a") failed: Property not found. retry in 120.298207ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1c6ee351-cab6-4c23-8658-c31bac40b40a
12222024-08-28T04:05:02.274ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4c8a8ded-ced6-466f-8626-7fd39c37d020") failed: Property not found. retry in 65.401383ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4c8a8ded-ced6-466f-8626-7fd39c37d020
12232024-08-28T04:05:02.287ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_bda2ed8e-2ea5-41eb-824b-d14527b76b8b") failed: Property not found. retry in 102.429479ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_bda2ed8e-2ea5-41eb-824b-d14527b76b8b
12242024-08-28T04:05:02.446ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_4eb364e4-14a2-46c4-b173-eb2890920e8f") failed: Property not found. retry in 1.183105525s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_4eb364e4-14a2-46c4-b173-eb2890920e8f
12252024-08-28T04:05:02.498ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1c6ee351-cab6-4c23-8658-c31bac40b40a") failed: Property not found. retry in 147.173288ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1c6ee351-cab6-4c23-8658-c31bac40b40a
12262024-08-28T04:05:02.524ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1d429812-8d5a-47a0-98f7-1f1f5ef6e285") failed: Property not found. retry in 1.348230909s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1d429812-8d5a-47a0-98f7-1f1f5ef6e285
12272024-08-28T04:05:02.538ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4c8a8ded-ced6-466f-8626-7fd39c37d020") failed: Property not found. retry in 135.762896ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4c8a8ded-ced6-466f-8626-7fd39c37d020
12282024-08-28T04:05:02.551ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_bda2ed8e-2ea5-41eb-824b-d14527b76b8b") failed: Property not found. retry in 124.664686ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_bda2ed8e-2ea5-41eb-824b-d14527b76b8b
12292024-08-28T04:05:02.561ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_24b1ac7b-c1d1-440a-a4e4-220c7ce34410") failed: Property not found. retry in 1.369832737s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_24b1ac7b-c1d1-440a-a4e4-220c7ce34410
12302024-08-28T04:05:02.562ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
12312024-08-28T04:05:02.661ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1c6ee351-cab6-4c23-8658-c31bac40b40a") failed: Property not found. retry in 289.722068ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1c6ee351-cab6-4c23-8658-c31bac40b40a
12322024-08-28T04:05:02.686ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_94b35d7a-af71-4078-a57c-874ff9a27068") failed: Property not found. retry in 1.054112134s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_94b35d7a-af71-4078-a57c-874ff9a27068
12332024-08-28T04:05:02.698ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_83266762-372c-41cd-a9dd-eee1b2c38bb6") failed: Property not found. retry in 1.021094437s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_83266762-372c-41cd-a9dd-eee1b2c38bb6
12342024-08-28T04:05:02.709ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4c8a8ded-ced6-466f-8626-7fd39c37d020") failed: Property not found. retry in 300.500872ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4c8a8ded-ced6-466f-8626-7fd39c37d020
12352024-08-28T04:05:02.720ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_bda2ed8e-2ea5-41eb-824b-d14527b76b8b") failed: Property not found. retry in 383.584661ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_bda2ed8e-2ea5-41eb-824b-d14527b76b8b
12362024-08-28T04:05:02.861ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7f6ebc53-fc17-488d-9b99-8dc983f4efec") failed: Property not found. retry in 670.490472ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7f6ebc53-fc17-488d-9b99-8dc983f4efec
12372024-08-28T04:05:02.873ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7e16ed01-11ff-45eb-8869-6ed06d9b0e70") failed: Property not found. retry in 648.124763ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7e16ed01-11ff-45eb-8869-6ed06d9b0e70
12382024-08-28T04:05:02.911ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d6ce85cc-4f25-4b5e-b0c0-a434368f63d4") failed: Property not found. retry in 640.238907ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d6ce85cc-4f25-4b5e-b0c0-a434368f63d4
12392024-08-28T04:05:02.964ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1c6ee351-cab6-4c23-8658-c31bac40b40a") failed: Property not found. retry in 861.028806ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1c6ee351-cab6-4c23-8658-c31bac40b40a
12402024-08-28T04:05:02.975ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d4f01887-9496-477a-9ed8-f93bf14fea73") failed: Property not found. retry in 554.594112ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d4f01887-9496-477a-9ed8-f93bf14fea73
12412024-08-28T04:05:03.021ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4c8a8ded-ced6-466f-8626-7fd39c37d020") failed: Property not found. retry in 650.335819ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4c8a8ded-ced6-466f-8626-7fd39c37d020
12422024-08-28T04:05:03.039ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_21378de4-15c3-48e2-b919-b1cc2363c4a6") failed: Property not found. retry in 706.885704ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_21378de4-15c3-48e2-b919-b1cc2363c4a6
12432024-08-28T04:05:03.100ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b839573-b270-4df0-b615-f92964eededd") failed: Property not found. retry in 1.261980759s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b839573-b270-4df0-b615-f92964eededd
12442024-08-28T04:05:03.115ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_bda2ed8e-2ea5-41eb-824b-d14527b76b8b") failed: Property not found. retry in 1.014354269s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_bda2ed8e-2ea5-41eb-824b-d14527b76b8b
12452024-08-28T04:05:03.137ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ff9eaaaf-6eec-47be-bc80-11d83c51090f") failed: Property not found. retry in 654.64217ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ff9eaaaf-6eec-47be-bc80-11d83c51090f
12462024-08-28T04:05:03.194ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d56672e9-8c1d-4f9a-a4b1-49fa3e1cfbe0") failed: Property not found. retry in 802.806669ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d56672e9-8c1d-4f9a-a4b1-49fa3e1cfbe0
12472024-08-28T04:05:03.279ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d073ae9b-afee-401e-bafb-45a05ad31f0d") failed: Property not found. retry in 686.735613ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d073ae9b-afee-401e-bafb-45a05ad31f0d
12482024-08-28T04:05:03.534ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7e16ed01-11ff-45eb-8869-6ed06d9b0e70") failed: Property not found. retry in 504.830787ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7e16ed01-11ff-45eb-8869-6ed06d9b0e70
12492024-08-28T04:05:03.547ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d4f01887-9496-477a-9ed8-f93bf14fea73") failed: Property not found. retry in 1.094375959s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d4f01887-9496-477a-9ed8-f93bf14fea73
12502024-08-28T04:05:03.560ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7f6ebc53-fc17-488d-9b99-8dc983f4efec") failed: Property not found. retry in 1.082158112s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7f6ebc53-fc17-488d-9b99-8dc983f4efec
12512024-08-28T04:05:03.573ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d6ce85cc-4f25-4b5e-b0c0-a434368f63d4") failed: Property not found. retry in 804.460409ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d6ce85cc-4f25-4b5e-b0c0-a434368f63d4
12522024-08-28T04:05:03.589ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a8d89411-e3d4-4290-8995-191bd8cdd564") failed: Property not found. retry in 953.810917ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a8d89411-e3d4-4290-8995-191bd8cdd564
12532024-08-28T04:05:03.642ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_4eb364e4-14a2-46c4-b173-eb2890920e8f") failed: Property not found. retry in 722.130334ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_4eb364e4-14a2-46c4-b173-eb2890920e8f
12542024-08-28T04:05:03.686ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4c8a8ded-ced6-466f-8626-7fd39c37d020") failed: Property not found. retry in 1.438014727s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4c8a8ded-ced6-466f-8626-7fd39c37d020
12552024-08-28T04:05:03.735ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_83266762-372c-41cd-a9dd-eee1b2c38bb6") failed: Property not found. retry in 1.221866729s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_83266762-372c-41cd-a9dd-eee1b2c38bb6
12562024-08-28T04:05:03.752ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_94b35d7a-af71-4078-a57c-874ff9a27068") failed: Property not found. retry in 659.034012ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_94b35d7a-af71-4078-a57c-874ff9a27068
12572024-08-28T04:05:03.764ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_21378de4-15c3-48e2-b919-b1cc2363c4a6") failed: Property not found. retry in 1.031949195s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_21378de4-15c3-48e2-b919-b1cc2363c4a6
12582024-08-28T04:05:03.806ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ff9eaaaf-6eec-47be-bc80-11d83c51090f") failed: Property not found. retry in 907.144252ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ff9eaaaf-6eec-47be-bc80-11d83c51090f
12592024-08-28T04:05:03.839ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1c6ee351-cab6-4c23-8658-c31bac40b40a") failed: Property not found. retry in 909.12413ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1c6ee351-cab6-4c23-8658-c31bac40b40a
12602024-08-28T04:05:03.886ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1d429812-8d5a-47a0-98f7-1f1f5ef6e285") failed: Property not found. retry in 815.447635ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1d429812-8d5a-47a0-98f7-1f1f5ef6e285
12612024-08-28T04:05:03.945ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_24b1ac7b-c1d1-440a-a4e4-220c7ce34410") failed: Property not found. retry in 891.768992ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_24b1ac7b-c1d1-440a-a4e4-220c7ce34410
12622024-08-28T04:05:03.980ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d073ae9b-afee-401e-bafb-45a05ad31f0d") failed: Property not found. retry in 1.069647092s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d073ae9b-afee-401e-bafb-45a05ad31f0d
12632024-08-28T04:05:04.011ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d56672e9-8c1d-4f9a-a4b1-49fa3e1cfbe0") failed: Property not found. retry in 1.233896544s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d56672e9-8c1d-4f9a-a4b1-49fa3e1cfbe0
12642024-08-28T04:05:04.052ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7e16ed01-11ff-45eb-8869-6ed06d9b0e70") failed: Property not found. retry in 1.002656882s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7e16ed01-11ff-45eb-8869-6ed06d9b0e70
12652024-08-28T04:05:04.143ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_bda2ed8e-2ea5-41eb-824b-d14527b76b8b") failed: Property not found. retry in 1.312766151s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_bda2ed8e-2ea5-41eb-824b-d14527b76b8b
12662024-08-28T04:05:04.376ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b839573-b270-4df0-b615-f92964eededd") failed: Property not found. retry in 1.07409637s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b839573-b270-4df0-b615-f92964eededd
12672024-08-28T04:05:04.386ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_4eb364e4-14a2-46c4-b173-eb2890920e8f") failed: Property not found. retry in 838.173927ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_4eb364e4-14a2-46c4-b173-eb2890920e8f
12682024-08-28T04:05:04.396ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d6ce85cc-4f25-4b5e-b0c0-a434368f63d4") failed: Property not found. retry in 875.520237ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d6ce85cc-4f25-4b5e-b0c0-a434368f63d4
12692024-08-28T04:05:04.423ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_94b35d7a-af71-4078-a57c-874ff9a27068") failed: Property not found. retry in 617.212753ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_94b35d7a-af71-4078-a57c-874ff9a27068
12702024-08-28T04:05:04.557ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a8d89411-e3d4-4290-8995-191bd8cdd564") failed: Property not found. retry in 1.390573012s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a8d89411-e3d4-4290-8995-191bd8cdd564
12712024-08-28T04:05:04.656ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d4f01887-9496-477a-9ed8-f93bf14fea73") failed: Property not found. retry in 787.46703ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d4f01887-9496-477a-9ed8-f93bf14fea73
12722024-08-28T04:05:04.673ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7f6ebc53-fc17-488d-9b99-8dc983f4efec") failed: Property not found. retry in 1.449419448s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7f6ebc53-fc17-488d-9b99-8dc983f4efec
12732024-08-28T04:05:04.720ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1d429812-8d5a-47a0-98f7-1f1f5ef6e285") failed: Property not found. retry in 698.946576ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1d429812-8d5a-47a0-98f7-1f1f5ef6e285
12742024-08-28T04:05:04.734ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ff9eaaaf-6eec-47be-bc80-11d83c51090f") failed: Property not found. retry in 1.331458686s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ff9eaaaf-6eec-47be-bc80-11d83c51090f
12752024-08-28T04:05:04.762ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1c6ee351-cab6-4c23-8658-c31bac40b40a") failed: Property not found. retry in 1.132822451s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1c6ee351-cab6-4c23-8658-c31bac40b40a
12762024-08-28T04:05:04.814ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_21378de4-15c3-48e2-b919-b1cc2363c4a6") failed: Property not found. retry in 1.146589621s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_21378de4-15c3-48e2-b919-b1cc2363c4a6
12772024-08-28T04:05:04.853ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_24b1ac7b-c1d1-440a-a4e4-220c7ce34410") failed: Property not found. retry in 1.38313256s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_24b1ac7b-c1d1-440a-a4e4-220c7ce34410
12782024-08-28T04:05:04.971ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_83266762-372c-41cd-a9dd-eee1b2c38bb6") failed: Property not found. retry in 1.001425611s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_83266762-372c-41cd-a9dd-eee1b2c38bb6
12792024-08-28T04:05:05.057ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_94b35d7a-af71-4078-a57c-874ff9a27068") failed: Property not found. retry in 1.452022085s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_94b35d7a-af71-4078-a57c-874ff9a27068
12802024-08-28T04:05:05.071ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d073ae9b-afee-401e-bafb-45a05ad31f0d") failed: Property not found. retry in 629.810586ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d073ae9b-afee-401e-bafb-45a05ad31f0d
12812024-08-28T04:05:05.081ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7e16ed01-11ff-45eb-8869-6ed06d9b0e70") failed: Property not found. retry in 794.883031ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7e16ed01-11ff-45eb-8869-6ed06d9b0e70
12822024-08-28T04:05:05.140ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4c8a8ded-ced6-466f-8626-7fd39c37d020") failed: Property not found. retry in 1.291671832s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4c8a8ded-ced6-466f-8626-7fd39c37d020
12832024-08-28T04:05:05.238ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_4eb364e4-14a2-46c4-b173-eb2890920e8f") failed: Property not found. retry in 1.101086303s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_4eb364e4-14a2-46c4-b173-eb2890920e8f
12842024-08-28T04:05:05.259ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d56672e9-8c1d-4f9a-a4b1-49fa3e1cfbe0") failed: Property not found. retry in 664.266507ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d56672e9-8c1d-4f9a-a4b1-49fa3e1cfbe0
12852024-08-28T04:05:05.288ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d6ce85cc-4f25-4b5e-b0c0-a434368f63d4") failed: Property not found. retry in 1.319172745s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d6ce85cc-4f25-4b5e-b0c0-a434368f63d4
12862024-08-28T04:05:05.433ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1d429812-8d5a-47a0-98f7-1f1f5ef6e285") failed: Property not found. retry in 1.09006915s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1d429812-8d5a-47a0-98f7-1f1f5ef6e285
12872024-08-28T04:05:05.465ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d4f01887-9496-477a-9ed8-f93bf14fea73") failed: Property not found. retry in 539.799469ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d4f01887-9496-477a-9ed8-f93bf14fea73
12882024-08-28T04:05:05.485ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b839573-b270-4df0-b615-f92964eededd") failed: Property not found. retry in 1.07419503s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b839573-b270-4df0-b615-f92964eededd
12892024-08-28T04:05:05.502ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_bda2ed8e-2ea5-41eb-824b-d14527b76b8b") failed: Property not found. retry in 826.52889ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_bda2ed8e-2ea5-41eb-824b-d14527b76b8b
12902024-08-28T04:05:05.722ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d073ae9b-afee-401e-bafb-45a05ad31f0d") failed: Property not found. retry in 715.579759ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d073ae9b-afee-401e-bafb-45a05ad31f0d
12912024-08-28T04:05:05.891ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7e16ed01-11ff-45eb-8869-6ed06d9b0e70") failed: Property not found. retry in 722.899359ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7e16ed01-11ff-45eb-8869-6ed06d9b0e70
12922024-08-28T04:05:05.910ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1c6ee351-cab6-4c23-8658-c31bac40b40a") failed: Property not found. retry in 1.168394786s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1c6ee351-cab6-4c23-8658-c31bac40b40a
12932024-08-28T04:05:05.941ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d56672e9-8c1d-4f9a-a4b1-49fa3e1cfbe0") failed: Property not found. retry in 528.668996ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d56672e9-8c1d-4f9a-a4b1-49fa3e1cfbe0
12942024-08-28T04:05:05.963ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a8d89411-e3d4-4290-8995-191bd8cdd564") failed: Property not found. retry in 892.355479ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a8d89411-e3d4-4290-8995-191bd8cdd564
12952024-08-28T04:05:05.978ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_21378de4-15c3-48e2-b919-b1cc2363c4a6") failed: Property not found. retry in 1.138023624s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_21378de4-15c3-48e2-b919-b1cc2363c4a6
12962024-08-28T04:05:05.993ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_83266762-372c-41cd-a9dd-eee1b2c38bb6") failed: Property not found. retry in 846.360435ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_83266762-372c-41cd-a9dd-eee1b2c38bb6
12972024-08-28T04:05:06.020ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d4f01887-9496-477a-9ed8-f93bf14fea73") failed: Property not found. retry in 1.236772284s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d4f01887-9496-477a-9ed8-f93bf14fea73
12982024-08-28T04:05:06.081ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ff9eaaaf-6eec-47be-bc80-11d83c51090f") failed: Property not found. retry in 1.29878981s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ff9eaaaf-6eec-47be-bc80-11d83c51090f
12992024-08-28T04:05:06.139ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7f6ebc53-fc17-488d-9b99-8dc983f4efec") failed: Property not found. retry in 1.27329329s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7f6ebc53-fc17-488d-9b99-8dc983f4efec
13002024-08-28T04:05:06.223ZINFOSledAgent (ServiceManager): Profile for oxz_clickhouse_f62eaa35-1217-424b-a535-2aa55c06fbad: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService18'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::e'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/clickhouse"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::e'/> <propval type="astring" name="listen_port" value='8123'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:1::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/clickhouse-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="clickhouse_address" value='[fd00:1122:3344:101::e]:8123'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::e]:8888'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13012024-08-28T04:05:06.224ZINFOSledAgent (ServiceManager): Booting oxz_clickhouse_f62eaa35-1217-424b-a535-2aa55c06fbad zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_clickhouse_f62eaa35-1217-424b-a535-2aa55c06fbad
13022024-08-28T04:05:06.258ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_24b1ac7b-c1d1-440a-a4e4-220c7ce34410") failed: Property not found. retry in 527.484746ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_24b1ac7b-c1d1-440a-a4e4-220c7ce34410
13032024-08-28T04:05:06.342ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_bda2ed8e-2ea5-41eb-824b-d14527b76b8b") failed: Property not found. retry in 1.377603845s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_bda2ed8e-2ea5-41eb-824b-d14527b76b8b
13042024-08-28T04:05:06.356ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_4eb364e4-14a2-46c4-b173-eb2890920e8f") failed: Property not found. retry in 1.485087241s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_4eb364e4-14a2-46c4-b173-eb2890920e8f
13052024-08-28T04:05:06.484ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4c8a8ded-ced6-466f-8626-7fd39c37d020") failed: Property not found. retry in 1.036198822s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4c8a8ded-ced6-466f-8626-7fd39c37d020
13062024-08-28T04:05:06.581ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d073ae9b-afee-401e-bafb-45a05ad31f0d") failed: Property not found. retry in 783.625293ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d073ae9b-afee-401e-bafb-45a05ad31f0d
13072024-08-28T04:05:06.705ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d56672e9-8c1d-4f9a-a4b1-49fa3e1cfbe0") failed: Property not found. retry in 624.989573ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d56672e9-8c1d-4f9a-a4b1-49fa3e1cfbe0
13082024-08-28T04:05:06.718ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d6ce85cc-4f25-4b5e-b0c0-a434368f63d4") failed: Property not found. retry in 1.292430282s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d6ce85cc-4f25-4b5e-b0c0-a434368f63d4
13092024-08-28T04:05:06.728ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b839573-b270-4df0-b615-f92964eededd") failed: Property not found. retry in 1.376253522s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b839573-b270-4df0-b615-f92964eededd
13102024-08-28T04:05:06.739ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_94b35d7a-af71-4078-a57c-874ff9a27068") failed: Property not found. retry in 1.348819963s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_94b35d7a-af71-4078-a57c-874ff9a27068
13112024-08-28T04:05:06.751ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7e16ed01-11ff-45eb-8869-6ed06d9b0e70") failed: Property not found. retry in 632.826864ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7e16ed01-11ff-45eb-8869-6ed06d9b0e70
13122024-08-28T04:05:06.763ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1d429812-8d5a-47a0-98f7-1f1f5ef6e285") failed: Property not found. retry in 1.175777717s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1d429812-8d5a-47a0-98f7-1f1f5ef6e285
13132024-08-28T04:05:06.786ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_f62eaa35-1217-424b-a535-2aa55c06fbad") failed: Property not found. retry in 50.803288ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_f62eaa35-1217-424b-a535-2aa55c06fbad
13142024-08-28T04:05:06.796ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_24b1ac7b-c1d1-440a-a4e4-220c7ce34410") failed: Property not found. retry in 907.638016ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_24b1ac7b-c1d1-440a-a4e4-220c7ce34410
13152024-08-28T04:05:06.848ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_f62eaa35-1217-424b-a535-2aa55c06fbad") failed: Property not found. retry in 127.877741ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_f62eaa35-1217-424b-a535-2aa55c06fbad
13162024-08-28T04:05:06.858ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_83266762-372c-41cd-a9dd-eee1b2c38bb6") failed: Property not found. retry in 1.039650608s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_83266762-372c-41cd-a9dd-eee1b2c38bb6
13172024-08-28T04:05:06.868ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a8d89411-e3d4-4290-8995-191bd8cdd564") failed: Property not found. retry in 923.751652ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a8d89411-e3d4-4290-8995-191bd8cdd564
13182024-08-28T04:05:06.988ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_f62eaa35-1217-424b-a535-2aa55c06fbad") failed: Property not found. retry in 237.296222ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_f62eaa35-1217-424b-a535-2aa55c06fbad
13192024-08-28T04:05:07.093ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1c6ee351-cab6-4c23-8658-c31bac40b40a") failed: Property not found. retry in 1.042978564s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1c6ee351-cab6-4c23-8658-c31bac40b40a
13202024-08-28T04:05:07.129ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_21378de4-15c3-48e2-b919-b1cc2363c4a6") failed: Property not found. retry in 896.702423ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_21378de4-15c3-48e2-b919-b1cc2363c4a6
13212024-08-28T04:05:07.238ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_f62eaa35-1217-424b-a535-2aa55c06fbad") failed: Property not found. retry in 351.424537ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_f62eaa35-1217-424b-a535-2aa55c06fbad
13222024-08-28T04:05:07.267ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d4f01887-9496-477a-9ed8-f93bf14fea73") failed: Property not found. retry in 597.095454ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d4f01887-9496-477a-9ed8-f93bf14fea73
13232024-08-28T04:05:07.341ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d56672e9-8c1d-4f9a-a4b1-49fa3e1cfbe0") failed: Property not found. retry in 678.854402ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d56672e9-8c1d-4f9a-a4b1-49fa3e1cfbe0
13242024-08-28T04:05:07.374ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d073ae9b-afee-401e-bafb-45a05ad31f0d") failed: Property not found. retry in 982.566145ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d073ae9b-afee-401e-bafb-45a05ad31f0d
13252024-08-28T04:05:07.391ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ff9eaaaf-6eec-47be-bc80-11d83c51090f") failed: Property not found. retry in 901.643749ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ff9eaaaf-6eec-47be-bc80-11d83c51090f
13262024-08-28T04:05:07.400ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7e16ed01-11ff-45eb-8869-6ed06d9b0e70") failed: Property not found. retry in 619.614961ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7e16ed01-11ff-45eb-8869-6ed06d9b0e70
13272024-08-28T04:05:07.422ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7f6ebc53-fc17-488d-9b99-8dc983f4efec") failed: Property not found. retry in 830.061929ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7f6ebc53-fc17-488d-9b99-8dc983f4efec
13282024-08-28T04:05:07.531ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4c8a8ded-ced6-466f-8626-7fd39c37d020") failed: Property not found. retry in 607.087967ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4c8a8ded-ced6-466f-8626-7fd39c37d020
13292024-08-28T04:05:07.606ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_f62eaa35-1217-424b-a535-2aa55c06fbad") failed: Property not found. retry in 533.197797ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_f62eaa35-1217-424b-a535-2aa55c06fbad
13302024-08-28T04:05:07.718ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_24b1ac7b-c1d1-440a-a4e4-220c7ce34410") failed: Property not found. retry in 1.441562096s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_24b1ac7b-c1d1-440a-a4e4-220c7ce34410
13312024-08-28T04:05:07.735ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_bda2ed8e-2ea5-41eb-824b-d14527b76b8b") failed: Property not found. retry in 830.483397ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_bda2ed8e-2ea5-41eb-824b-d14527b76b8b
13322024-08-28T04:05:07.803ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a8d89411-e3d4-4290-8995-191bd8cdd564") failed: Property not found. retry in 1.458238766s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a8d89411-e3d4-4290-8995-191bd8cdd564
13332024-08-28T04:05:07.852ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_4eb364e4-14a2-46c4-b173-eb2890920e8f") failed: Property not found. retry in 858.818866ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_4eb364e4-14a2-46c4-b173-eb2890920e8f
13342024-08-28T04:05:07.876ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d4f01887-9496-477a-9ed8-f93bf14fea73") failed: Property not found. retry in 971.874021ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d4f01887-9496-477a-9ed8-f93bf14fea73
13352024-08-28T04:05:07.910ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_83266762-372c-41cd-a9dd-eee1b2c38bb6") failed: Property not found. retry in 638.369569ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_83266762-372c-41cd-a9dd-eee1b2c38bb6
13362024-08-28T04:05:07.952ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1d429812-8d5a-47a0-98f7-1f1f5ef6e285") failed: Property not found. retry in 879.235163ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1d429812-8d5a-47a0-98f7-1f1f5ef6e285
13372024-08-28T04:05:08.023ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d6ce85cc-4f25-4b5e-b0c0-a434368f63d4") failed: Property not found. retry in 1.402972261s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d6ce85cc-4f25-4b5e-b0c0-a434368f63d4
13382024-08-28T04:05:08.035ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7e16ed01-11ff-45eb-8869-6ed06d9b0e70") failed: Property not found. retry in 974.340136ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7e16ed01-11ff-45eb-8869-6ed06d9b0e70
13392024-08-28T04:05:08.044ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d56672e9-8c1d-4f9a-a4b1-49fa3e1cfbe0") failed: Property not found. retry in 1.296577901s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d56672e9-8c1d-4f9a-a4b1-49fa3e1cfbe0
13402024-08-28T04:05:08.054ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_21378de4-15c3-48e2-b919-b1cc2363c4a6") failed: Property not found. retry in 890.491431ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_21378de4-15c3-48e2-b919-b1cc2363c4a6
13412024-08-28T04:05:08.101ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_94b35d7a-af71-4078-a57c-874ff9a27068") failed: Property not found. retry in 1.412049909s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_94b35d7a-af71-4078-a57c-874ff9a27068
13422024-08-28T04:05:08.116ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b839573-b270-4df0-b615-f92964eededd") failed: Property not found. retry in 893.514497ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b839573-b270-4df0-b615-f92964eededd
13432024-08-28T04:05:08.149ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1c6ee351-cab6-4c23-8658-c31bac40b40a") failed: Property not found. retry in 1.000123033s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1c6ee351-cab6-4c23-8658-c31bac40b40a
13442024-08-28T04:05:08.161ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_f62eaa35-1217-424b-a535-2aa55c06fbad") failed: Property not found. retry in 1.403484733s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_f62eaa35-1217-424b-a535-2aa55c06fbad
13452024-08-28T04:05:08.173ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4c8a8ded-ced6-466f-8626-7fd39c37d020") failed: Property not found. retry in 1.295266562s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4c8a8ded-ced6-466f-8626-7fd39c37d020
13462024-08-28T04:05:08.265ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7f6ebc53-fc17-488d-9b99-8dc983f4efec") failed: Property not found. retry in 1.326885864s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7f6ebc53-fc17-488d-9b99-8dc983f4efec
13472024-08-28T04:05:08.308ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ff9eaaaf-6eec-47be-bc80-11d83c51090f") failed: Property not found. retry in 1.479622258s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ff9eaaaf-6eec-47be-bc80-11d83c51090f
13482024-08-28T04:05:08.373ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d073ae9b-afee-401e-bafb-45a05ad31f0d") failed: Property not found. retry in 1.185938294s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d073ae9b-afee-401e-bafb-45a05ad31f0d
13492024-08-28T04:05:08.563ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_83266762-372c-41cd-a9dd-eee1b2c38bb6") failed: Property not found. retry in 975.385997ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_83266762-372c-41cd-a9dd-eee1b2c38bb6
13502024-08-28T04:05:08.580ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_bda2ed8e-2ea5-41eb-824b-d14527b76b8b") failed: Property not found. retry in 630.792251ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_bda2ed8e-2ea5-41eb-824b-d14527b76b8b
13512024-08-28T04:05:08.731ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_4eb364e4-14a2-46c4-b173-eb2890920e8f") failed: Property not found. retry in 773.833387ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_4eb364e4-14a2-46c4-b173-eb2890920e8f
13522024-08-28T04:05:08.843ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1d429812-8d5a-47a0-98f7-1f1f5ef6e285") failed: Property not found. retry in 769.415796ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1d429812-8d5a-47a0-98f7-1f1f5ef6e285
13532024-08-28T04:05:08.864ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d4f01887-9496-477a-9ed8-f93bf14fea73") failed: Property not found. retry in 1.249908155s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d4f01887-9496-477a-9ed8-f93bf14fea73
13542024-08-28T04:05:08.865ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:54247
13552024-08-28T04:05:08.880ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 13162 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:54247 req_id = 5cc2e6d7-9939-4645-b0e8-23a8eaeb2f8f response_code = 200 uri = /baseboard
13562024-08-28T04:05:08.960ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_21378de4-15c3-48e2-b919-b1cc2363c4a6") failed: Property not found. retry in 778.963839ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_21378de4-15c3-48e2-b919-b1cc2363c4a6
13572024-08-28T04:05:09.025ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b839573-b270-4df0-b615-f92964eededd") failed: Property not found. retry in 1.040566738s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b839573-b270-4df0-b615-f92964eededd
13582024-08-28T04:05:09.038ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7e16ed01-11ff-45eb-8869-6ed06d9b0e70") failed: Property not found. retry in 960.139851ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7e16ed01-11ff-45eb-8869-6ed06d9b0e70
13592024-08-28T04:05:09.164ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1c6ee351-cab6-4c23-8658-c31bac40b40a") failed: Property not found. retry in 838.640167ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1c6ee351-cab6-4c23-8658-c31bac40b40a
13602024-08-28T04:05:09.178ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_24b1ac7b-c1d1-440a-a4e4-220c7ce34410") failed: Property not found. retry in 1.315142367s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_24b1ac7b-c1d1-440a-a4e4-220c7ce34410
13612024-08-28T04:05:09.223ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_bda2ed8e-2ea5-41eb-824b-d14527b76b8b") failed: Property not found. retry in 1.174172451s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_bda2ed8e-2ea5-41eb-824b-d14527b76b8b
13622024-08-28T04:05:09.273ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a8d89411-e3d4-4290-8995-191bd8cdd564") failed: Property not found. retry in 1.281060117s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a8d89411-e3d4-4290-8995-191bd8cdd564
13632024-08-28T04:05:09.356ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d56672e9-8c1d-4f9a-a4b1-49fa3e1cfbe0") failed: Property not found. retry in 1.012573825s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d56672e9-8c1d-4f9a-a4b1-49fa3e1cfbe0
13642024-08-28T04:05:09.438ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d6ce85cc-4f25-4b5e-b0c0-a434368f63d4") failed: Property not found. retry in 679.157965ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d6ce85cc-4f25-4b5e-b0c0-a434368f63d4
13652024-08-28T04:05:09.482ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4c8a8ded-ced6-466f-8626-7fd39c37d020") failed: Property not found. retry in 1.140378672s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4c8a8ded-ced6-466f-8626-7fd39c37d020
13662024-08-28T04:05:09.519ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_4eb364e4-14a2-46c4-b173-eb2890920e8f") failed: Property not found. retry in 863.381958ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_4eb364e4-14a2-46c4-b173-eb2890920e8f
13672024-08-28T04:05:09.537ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_94b35d7a-af71-4078-a57c-874ff9a27068") failed: Property not found. retry in 1.438467033s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_94b35d7a-af71-4078-a57c-874ff9a27068
13682024-08-28T04:05:09.554ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_83266762-372c-41cd-a9dd-eee1b2c38bb6") failed: Property not found. retry in 1.043557753s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_83266762-372c-41cd-a9dd-eee1b2c38bb6
13692024-08-28T04:05:09.575ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d073ae9b-afee-401e-bafb-45a05ad31f0d") failed: Property not found. retry in 715.885773ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d073ae9b-afee-401e-bafb-45a05ad31f0d
13702024-08-28T04:05:09.588ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_f62eaa35-1217-424b-a535-2aa55c06fbad") failed: Property not found. retry in 1.313362257s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_f62eaa35-1217-424b-a535-2aa55c06fbad
13712024-08-28T04:05:09.607ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7f6ebc53-fc17-488d-9b99-8dc983f4efec") failed: Property not found. retry in 969.461904ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7f6ebc53-fc17-488d-9b99-8dc983f4efec
13722024-08-28T04:05:09.627ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1d429812-8d5a-47a0-98f7-1f1f5ef6e285") failed: Property not found. retry in 789.909422ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1d429812-8d5a-47a0-98f7-1f1f5ef6e285
13732024-08-28T04:05:09.756ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_21378de4-15c3-48e2-b919-b1cc2363c4a6") failed: Property not found. retry in 773.650174ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_21378de4-15c3-48e2-b919-b1cc2363c4a6
13742024-08-28T04:05:09.806ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ff9eaaaf-6eec-47be-bc80-11d83c51090f") failed: Property not found. retry in 671.429179ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ff9eaaaf-6eec-47be-bc80-11d83c51090f
13752024-08-28T04:05:10.014ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7e16ed01-11ff-45eb-8869-6ed06d9b0e70") failed: Property not found. retry in 1.166338438s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7e16ed01-11ff-45eb-8869-6ed06d9b0e70
13762024-08-28T04:05:10.031ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1c6ee351-cab6-4c23-8658-c31bac40b40a") failed: Property not found. retry in 1.174368871s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1c6ee351-cab6-4c23-8658-c31bac40b40a
13772024-08-28T04:05:10.081ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b839573-b270-4df0-b615-f92964eededd") failed: Property not found. retry in 1.331303927s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b839573-b270-4df0-b615-f92964eededd
13782024-08-28T04:05:10.126ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d4f01887-9496-477a-9ed8-f93bf14fea73") failed: Property not found. retry in 1.41608323s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d4f01887-9496-477a-9ed8-f93bf14fea73
13792024-08-28T04:05:10.135ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d6ce85cc-4f25-4b5e-b0c0-a434368f63d4") failed: Property not found. retry in 727.72229ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d6ce85cc-4f25-4b5e-b0c0-a434368f63d4
13802024-08-28T04:05:10.302ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d073ae9b-afee-401e-bafb-45a05ad31f0d") failed: Property not found. retry in 1.314740504s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d073ae9b-afee-401e-bafb-45a05ad31f0d
13812024-08-28T04:05:10.380ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d56672e9-8c1d-4f9a-a4b1-49fa3e1cfbe0") failed: Property not found. retry in 539.679058ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d56672e9-8c1d-4f9a-a4b1-49fa3e1cfbe0
13822024-08-28T04:05:10.394ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_4eb364e4-14a2-46c4-b173-eb2890920e8f") failed: Property not found. retry in 1.337828263s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_4eb364e4-14a2-46c4-b173-eb2890920e8f
13832024-08-28T04:05:10.408ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_bda2ed8e-2ea5-41eb-824b-d14527b76b8b") failed: Property not found. retry in 809.413329ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_bda2ed8e-2ea5-41eb-824b-d14527b76b8b
13842024-08-28T04:05:10.429ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1d429812-8d5a-47a0-98f7-1f1f5ef6e285") failed: Property not found. retry in 1.117542473s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1d429812-8d5a-47a0-98f7-1f1f5ef6e285
13852024-08-28T04:05:10.487ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ff9eaaaf-6eec-47be-bc80-11d83c51090f") failed: Property not found. retry in 679.715599ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ff9eaaaf-6eec-47be-bc80-11d83c51090f
13862024-08-28T04:05:10.502ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_24b1ac7b-c1d1-440a-a4e4-220c7ce34410") failed: Property not found. retry in 808.014372ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_24b1ac7b-c1d1-440a-a4e4-220c7ce34410
13872024-08-28T04:05:10.538ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_21378de4-15c3-48e2-b919-b1cc2363c4a6") failed: Property not found. retry in 713.150021ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_21378de4-15c3-48e2-b919-b1cc2363c4a6
13882024-08-28T04:05:10.563ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a8d89411-e3d4-4290-8995-191bd8cdd564") failed: Property not found. retry in 742.588583ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a8d89411-e3d4-4290-8995-191bd8cdd564
13892024-08-28T04:05:10.585ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7f6ebc53-fc17-488d-9b99-8dc983f4efec") failed: Property not found. retry in 854.436839ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7f6ebc53-fc17-488d-9b99-8dc983f4efec
13902024-08-28T04:05:10.606ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_83266762-372c-41cd-a9dd-eee1b2c38bb6") failed: Property not found. retry in 1.246427025s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_83266762-372c-41cd-a9dd-eee1b2c38bb6
13912024-08-28T04:05:10.633ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4c8a8ded-ced6-466f-8626-7fd39c37d020") failed: Property not found. retry in 952.084105ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4c8a8ded-ced6-466f-8626-7fd39c37d020
13922024-08-28T04:05:10.876ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d6ce85cc-4f25-4b5e-b0c0-a434368f63d4") failed: Property not found. retry in 1.259554115s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d6ce85cc-4f25-4b5e-b0c0-a434368f63d4
13932024-08-28T04:05:10.911ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_f62eaa35-1217-424b-a535-2aa55c06fbad") failed: Property not found. retry in 1.246873981s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_f62eaa35-1217-424b-a535-2aa55c06fbad
13942024-08-28T04:05:10.931ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d56672e9-8c1d-4f9a-a4b1-49fa3e1cfbe0") failed: Property not found. retry in 565.715393ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d56672e9-8c1d-4f9a-a4b1-49fa3e1cfbe0
13952024-08-28T04:05:10.986ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_94b35d7a-af71-4078-a57c-874ff9a27068") failed: Property not found. retry in 1.28264817s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_94b35d7a-af71-4078-a57c-874ff9a27068
13962024-08-28T04:05:11.178ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ff9eaaaf-6eec-47be-bc80-11d83c51090f") failed: Property not found. retry in 1.350367149s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ff9eaaaf-6eec-47be-bc80-11d83c51090f
13972024-08-28T04:05:11.191ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7e16ed01-11ff-45eb-8869-6ed06d9b0e70") failed: Property not found. retry in 518.525909ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7e16ed01-11ff-45eb-8869-6ed06d9b0e70
13982024-08-28T04:05:11.215ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1c6ee351-cab6-4c23-8658-c31bac40b40a") failed: Property not found. retry in 1.358623761s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1c6ee351-cab6-4c23-8658-c31bac40b40a
13992024-08-28T04:05:11.228ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_bda2ed8e-2ea5-41eb-824b-d14527b76b8b") failed: Property not found. retry in 673.864563ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_bda2ed8e-2ea5-41eb-824b-d14527b76b8b
14002024-08-28T04:05:11.261ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_21378de4-15c3-48e2-b919-b1cc2363c4a6") failed: Property not found. retry in 1.202329005s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_21378de4-15c3-48e2-b919-b1cc2363c4a6
14012024-08-28T04:05:11.315ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a8d89411-e3d4-4290-8995-191bd8cdd564") failed: Property not found. retry in 1.422875839s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a8d89411-e3d4-4290-8995-191bd8cdd564
14022024-08-28T04:05:11.325ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_24b1ac7b-c1d1-440a-a4e4-220c7ce34410") failed: Property not found. retry in 1.486332156s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_24b1ac7b-c1d1-440a-a4e4-220c7ce34410
14032024-08-28T04:05:11.423ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b839573-b270-4df0-b615-f92964eededd") failed: Property not found. retry in 636.258625ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b839573-b270-4df0-b615-f92964eededd
14042024-08-28T04:05:11.449ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7f6ebc53-fc17-488d-9b99-8dc983f4efec") failed: Property not found. retry in 1.154981958s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7f6ebc53-fc17-488d-9b99-8dc983f4efec
14052024-08-28T04:05:11.552ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d4f01887-9496-477a-9ed8-f93bf14fea73") failed: Property not found. retry in 956.667823ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d4f01887-9496-477a-9ed8-f93bf14fea73
14062024-08-28T04:05:11.562ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1d429812-8d5a-47a0-98f7-1f1f5ef6e285") failed: Property not found. retry in 1.356016377s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1d429812-8d5a-47a0-98f7-1f1f5ef6e285
14072024-08-28T04:05:11.596ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4c8a8ded-ced6-466f-8626-7fd39c37d020") failed: Property not found. retry in 1.004252797s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4c8a8ded-ced6-466f-8626-7fd39c37d020
14082024-08-28T04:05:11.629ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d073ae9b-afee-401e-bafb-45a05ad31f0d") failed: Property not found. retry in 807.003414ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d073ae9b-afee-401e-bafb-45a05ad31f0d
14092024-08-28T04:05:11.719ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_7e16ed01-11ff-45eb-8869-6ed06d9b0e70") failed: Property not found. retry in 864.228931ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_7e16ed01-11ff-45eb-8869-6ed06d9b0e70
14102024-08-28T04:05:11.743ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_4eb364e4-14a2-46c4-b173-eb2890920e8f") failed: Property not found. retry in 535.497704ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_4eb364e4-14a2-46c4-b173-eb2890920e8f
14112024-08-28T04:05:11.915ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_bda2ed8e-2ea5-41eb-824b-d14527b76b8b") failed: Property not found. retry in 1.423232547s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_bda2ed8e-2ea5-41eb-824b-d14527b76b8b
14122024-08-28T04:05:12.170ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_f62eaa35-1217-424b-a535-2aa55c06fbad") failed: Property not found. retry in 639.980107ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_f62eaa35-1217-424b-a535-2aa55c06fbad
14132024-08-28T04:05:12.829ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_f62eaa35-1217-424b-a535-2aa55c06fbad") failed: Property not found. retry in 613.82045ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_f62eaa35-1217-424b-a535-2aa55c06fbad
14142024-08-28T04:05:13.463ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_f62eaa35-1217-424b-a535-2aa55c06fbad") failed: Property not found. retry in 640.853316ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_f62eaa35-1217-424b-a535-2aa55c06fbad
14152024-08-28T04:05:14.128ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_f62eaa35-1217-424b-a535-2aa55c06fbad") failed: Property not found. retry in 761.485226ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_f62eaa35-1217-424b-a535-2aa55c06fbad
14162024-08-28T04:05:14.916ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_f62eaa35-1217-424b-a535-2aa55c06fbad") failed: Property not found. retry in 1.474518162s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_f62eaa35-1217-424b-a535-2aa55c06fbad
14172024-08-28T04:05:16.422ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_crucible_d6ce85cc-4f25-4b5e-b0c0-a434368f63d4
14182024-08-28T04:05:16.422ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_crucible_d4f01887-9496-477a-9ed8-f93bf14fea73
14192024-08-28T04:05:16.422ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_crucible_4b839573-b270-4df0-b615-f92964eededd
14202024-08-28T04:05:16.422ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_oximeter_21378de4-15c3-48e2-b919-b1cc2363c4a6
14212024-08-28T04:05:16.422ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_crucible_d073ae9b-afee-401e-bafb-45a05ad31f0d
14222024-08-28T04:05:16.422ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_external_dns_4eb364e4-14a2-46c4-b173-eb2890920e8f
14232024-08-28T04:05:16.422ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_external_dns_94b35d7a-af71-4078-a57c-874ff9a27068
14242024-08-28T04:05:16.422ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_crucible_ff9eaaaf-6eec-47be-bc80-11d83c51090f
14252024-08-28T04:05:16.422ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_crucible_7e16ed01-11ff-45eb-8869-6ed06d9b0e70
14262024-08-28T04:05:16.422ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_crucible_a8d89411-e3d4-4290-8995-191bd8cdd564
14272024-08-28T04:05:16.422ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_nexus_1c6ee351-cab6-4c23-8658-c31bac40b40a
14282024-08-28T04:05:16.422ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_clickhouse_f62eaa35-1217-424b-a535-2aa55c06fbad
14292024-08-28T04:05:16.422ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_crucible_pantry_1d429812-8d5a-47a0-98f7-1f1f5ef6e285
14302024-08-28T04:05:16.422ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_crucible_pantry_83266762-372c-41cd-a9dd-eee1b2c38bb6
14312024-08-28T04:05:16.422ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_crucible_pantry_d56672e9-8c1d-4f9a-a4b1-49fa3e1cfbe0
14322024-08-28T04:05:16.422ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_nexus_4c8a8ded-ced6-466f-8626-7fd39c37d020
14332024-08-28T04:05:16.422ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_nexus_bda2ed8e-2ea5-41eb-824b-d14527b76b8b
14342024-08-28T04:05:16.422ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_crucible_7f6ebc53-fc17-488d-9b99-8dc983f4efec
14352024-08-28T04:05:16.422ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_crucible_24b1ac7b-c1d1-440a-a4e4-220c7ce34410
14362024-08-28T04:05:16.429ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/51625293-e5a8-4cc4-9640-b64176e7a48e/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
14372024-08-28T04:05:16.432ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/4dad4c76-eb6c-4281-9f1d-a63fdb0117e1/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
14382024-08-28T04:05:16.436ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 40631519 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:32833 req_id = fca59318-1e7d-4f96-861f-a7362a33aa33 response_code = 204 uri = /omicron-zones
14392024-08-28T04:05:16.448ZINFOSledAgent (RSS): Finished setting up services file = sled-agent/src/rack_setup/service.rs:1286
14402024-08-28T04:05:16.448ZINFOSledAgent (RSS): Writing ledger to /pool/int/51625293-e5a8-4cc4-9640-b64176e7a48e/config/.rss-plan-completed.marker.tmp file = common/src/ledger.rs:199
14412024-08-28T04:05:16.449ZINFOSledAgent (RSS): Writing ledger to /pool/int/4dad4c76-eb6c-4281-9f1d-a63fdb0117e1/config/.rss-plan-completed.marker.tmp file = common/src/ledger.rs:199
14422024-08-28T04:05:16.452ZINFOSledAgent (RSS): Handing off control to Nexus file = sled-agent/src/rack_setup/service.rs:687
14432024-08-28T04:05:16.457ZINFOSledAgent (RSS): Nexus address: [fd00:1122:3344:101::b]:12221 file = sled-agent/src/rack_setup/service.rs:700
14442024-08-28T04:05:16.493ZINFOSledAgent (RSS): rack_network_config: RackNetworkConfigV2 { bfd: [], bgp: [], infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [ PortConfigV2 { addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.2.174, width: 24, }, ), vlan_id: None, }, ], autoneg: false, bgp_peers: [], lldp: None, port: "qsfp0", routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), local_pref: None, nexthop: 10.151.2.100, vlan_id: None, }, ], switch: Switch0, uplink_port_fec: None, uplink_port_speed: Speed40G, }, ], rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, } file = sled-agent/src/rack_setup/service.rs:867
14452024-08-28T04:05:16.498ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/8b05f6f6-ff29-4d8c-aa09-514d34e6a880/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
14462024-08-28T04:05:16.619ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/8b05f6f6-ff29-4d8c-aa09-514d34e6a880/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
14472024-08-28T04:05:16.729ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/8b05f6f6-ff29-4d8c-aa09-514d34e6a880/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
14482024-08-28T04:05:16.931ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/8b05f6f6-ff29-4d8c-aa09-514d34e6a880/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
14492024-08-28T04:05:17.094ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/8b05f6f6-ff29-4d8c-aa09-514d34e6a880/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
14502024-08-28T04:05:17.369ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/8b05f6f6-ff29-4d8c-aa09-514d34e6a880/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
14512024-08-28T04:05:17.574ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/8b05f6f6-ff29-4d8c-aa09-514d34e6a880/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
14522024-08-28T04:05:17.768ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/8b05f6f6-ff29-4d8c-aa09-514d34e6a880/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
14532024-08-28T04:05:17.985ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/8b05f6f6-ff29-4d8c-aa09-514d34e6a880/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
14542024-08-28T04:05:18.286ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/8b05f6f6-ff29-4d8c-aa09-514d34e6a880/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
14552024-08-28T04:05:18.700ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/8b05f6f6-ff29-4d8c-aa09-514d34e6a880/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
14562024-08-28T04:05:19.361ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/8b05f6f6-ff29-4d8c-aa09-514d34e6a880/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
14572024-08-28T04:05:19.910ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/8b05f6f6-ff29-4d8c-aa09-514d34e6a880/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
14582024-08-28T04:05:21.089ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/8b05f6f6-ff29-4d8c-aa09-514d34e6a880/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
14592024-08-28T04:05:22.196ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/8b05f6f6-ff29-4d8c-aa09-514d34e6a880/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
14602024-08-28T04:05:22.910ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/8b05f6f6-ff29-4d8c-aa09-514d34e6a880/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
14612024-08-28T04:05:23.814ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/8b05f6f6-ff29-4d8c-aa09-514d34e6a880/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
14622024-08-28T04:05:25.445ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/8b05f6f6-ff29-4d8c-aa09-514d34e6a880/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
14632024-08-28T04:05:28.036ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/8b05f6f6-ff29-4d8c-aa09-514d34e6a880/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
14642024-08-28T04:05:30.184ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/8b05f6f6-ff29-4d8c-aa09-514d34e6a880/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
14652024-08-28T04:05:32.812ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/8b05f6f6-ff29-4d8c-aa09-514d34e6a880/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
14662024-08-28T04:05:38.436ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "f82edf30-4a79-4bee-a9cd-84139c6b3f10", "content-length": "173", "date": "Wed, 28 Aug 2024 04:05:37 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"f62e5a13-51f3-4379-acf3-2f8e8601ac1f\"", request_id: "f82edf30-4a79-4bee-a9cd-84139c6b3f10" } file = sled-agent/src/rack_setup/service.rs:929
14672024-08-28T04:05:38.924ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:53448
14682024-08-28T04:05:38.924ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 34 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:53448 req_id = 45d57331-c1cf-4785-939c-66bb265f1d78 response_code = 200 uri = /baseboard
14692024-08-28T04:05:43.750ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "127d7530-18eb-4855-918b-250e322e5c6e", "content-length": "173", "date": "Wed, 28 Aug 2024 04:05:43 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"f62e5a13-51f3-4379-acf3-2f8e8601ac1f\"", request_id: "127d7530-18eb-4855-918b-250e322e5c6e" } file = sled-agent/src/rack_setup/service.rs:929
14702024-08-28T04:05:44.349ZINFOSledAgent (NexusNotifierTask): Successfully put SledAgentInfo to nexus file = sled-agent/src/nexus.rs:544 new_generation = 1 old_generation = 1 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f
14712024-08-28T04:05:44.437ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:63364
14722024-08-28T04:05:44.440ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
14732024-08-28T04:05:44.526ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 88777 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:63364 req_id = 40b9bcf0-cde5-470d-bdbf-aef7a1b841de response_code = 200 uri = /inventory
14742024-08-28T04:05:44.527ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
14752024-08-28T04:05:44.527ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/51625293-e5a8-4cc4-9640-b64176e7a48e/config/omicron-zones.json file = common/src/ledger.rs:177
14762024-08-28T04:05:44.529ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/4dad4c76-eb6c-4281-9f1d-a63fdb0117e1/config/omicron-zones.json file = common/src/ledger.rs:177
14772024-08-28T04:05:44.530ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3104 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:63364 req_id = efd1599f-18c6-4afc-a586-2b510c88a584 response_code = 200 uri = /omicron-zones
14782024-08-28T04:05:50.360ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:33481
14792024-08-28T04:05:50.366ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 4b747939-586b-48cb-a488-7f7f972db309, kind: Service { id: 4eb364e4-14a2-46c4-b173-eb2890920e8f }, name: Name("external-dns-4eb364e4-14a2-46c4-b173-eb2890920e8f"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 246, 113])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a3e0f17f-d79d-4cc0-ba0f-997e844ad5f3, kind: Service { id: 94b35d7a-af71-4078-a57c-874ff9a27068 }, name: Name("external-dns-94b35d7a-af71-4078-a57c-874ff9a27068"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 165, 179])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0cddb53a-efd3-4f89-93d7-81b182af0cdb, kind: Service { id: 1c6ee351-cab6-4c23-8658-c31bac40b40a }, name: Name("nexus-1c6ee351-cab6-4c23-8658-c31bac40b40a"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 208, 245])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 1a83b6d1-627c-41ca-8268-a29f26800e91, kind: Service { id: 4c8a8ded-ced6-466f-8626-7fd39c37d020 }, name: Name("nexus-4c8a8ded-ced6-466f-8626-7fd39c37d020"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 220])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 34a49c6e-26fe-4abd-8702-695d279dbdca, kind: Service { id: bda2ed8e-2ea5-41eb-824b-d14527b76b8b }, name: Name("nexus-bda2ed8e-2ea5-41eb-824b-d14527b76b8b"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 210, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
14802024-08-28T04:05:50.367ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
14812024-08-28T04:05:50.367ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
14822024-08-28T04:05:50.367ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
14832024-08-28T04:05:50.367ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
14842024-08-28T04:05:50.367ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
14852024-08-28T04:05:50.367ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
14862024-08-28T04:05:50.367ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 6457 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:33481 req_id = fec00fc3-2df2-4a68-93bb-185faea482fe response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
14872024-08-28T04:05:50.368ZINFOSledAgent (RSS): Handoff to Nexus is complete file = sled-agent/src/rack_setup/service.rs:939
14882024-08-28T04:05:50.621ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:34413
14892024-08-28T04:05:50.621ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
14902024-08-28T04:05:50.719ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 97547 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:34413 req_id = b3ffe745-621c-42e6-a79c-ebda2cdd3c7c response_code = 200 uri = /inventory
14912024-08-28T04:05:50.719ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:33432
14922024-08-28T04:05:50.719ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 4b747939-586b-48cb-a488-7f7f972db309, kind: Service { id: 4eb364e4-14a2-46c4-b173-eb2890920e8f }, name: Name("external-dns-4eb364e4-14a2-46c4-b173-eb2890920e8f"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 246, 113])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a3e0f17f-d79d-4cc0-ba0f-997e844ad5f3, kind: Service { id: 94b35d7a-af71-4078-a57c-874ff9a27068 }, name: Name("external-dns-94b35d7a-af71-4078-a57c-874ff9a27068"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 165, 179])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0cddb53a-efd3-4f89-93d7-81b182af0cdb, kind: Service { id: 1c6ee351-cab6-4c23-8658-c31bac40b40a }, name: Name("nexus-1c6ee351-cab6-4c23-8658-c31bac40b40a"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 208, 245])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 1a83b6d1-627c-41ca-8268-a29f26800e91, kind: Service { id: 4c8a8ded-ced6-466f-8626-7fd39c37d020 }, name: Name("nexus-4c8a8ded-ced6-466f-8626-7fd39c37d020"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 220])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 34a49c6e-26fe-4abd-8702-695d279dbdca, kind: Service { id: bda2ed8e-2ea5-41eb-824b-d14527b76b8b }, name: Name("nexus-bda2ed8e-2ea5-41eb-824b-d14527b76b8b"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 210, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
14932024-08-28T04:05:50.719ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
14942024-08-28T04:05:50.719ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
14952024-08-28T04:05:50.719ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
14962024-08-28T04:05:50.719ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
14972024-08-28T04:05:50.719ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
14982024-08-28T04:05:50.719ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
14992024-08-28T04:05:50.719ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 262 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:33432 req_id = 0027f44a-c9e5-4935-ac74-a0a8ca3ce90f response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
15002024-08-28T04:05:50.720ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
15012024-08-28T04:05:50.720ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/51625293-e5a8-4cc4-9640-b64176e7a48e/config/omicron-zones.json file = common/src/ledger.rs:177
15022024-08-28T04:05:50.720ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/4dad4c76-eb6c-4281-9f1d-a63fdb0117e1/config/omicron-zones.json file = common/src/ledger.rs:177
15032024-08-28T04:05:50.721ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1605 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:34413 req_id = 3f80c3ec-2232-46a1-95df-25ecc47ad2b1 response_code = 200 uri = /omicron-zones
15042024-08-28T04:05:50.732ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:61872
15052024-08-28T04:05:50.733ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 4b747939-586b-48cb-a488-7f7f972db309, kind: Service { id: 4eb364e4-14a2-46c4-b173-eb2890920e8f }, name: Name("external-dns-4eb364e4-14a2-46c4-b173-eb2890920e8f"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 246, 113])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a3e0f17f-d79d-4cc0-ba0f-997e844ad5f3, kind: Service { id: 94b35d7a-af71-4078-a57c-874ff9a27068 }, name: Name("external-dns-94b35d7a-af71-4078-a57c-874ff9a27068"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 165, 179])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0cddb53a-efd3-4f89-93d7-81b182af0cdb, kind: Service { id: 1c6ee351-cab6-4c23-8658-c31bac40b40a }, name: Name("nexus-1c6ee351-cab6-4c23-8658-c31bac40b40a"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 208, 245])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 1a83b6d1-627c-41ca-8268-a29f26800e91, kind: Service { id: 4c8a8ded-ced6-466f-8626-7fd39c37d020 }, name: Name("nexus-4c8a8ded-ced6-466f-8626-7fd39c37d020"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 220])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 34a49c6e-26fe-4abd-8702-695d279dbdca, kind: Service { id: bda2ed8e-2ea5-41eb-824b-d14527b76b8b }, name: Name("nexus-bda2ed8e-2ea5-41eb-824b-d14527b76b8b"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 210, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
15062024-08-28T04:05:50.733ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
15072024-08-28T04:05:50.733ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
15082024-08-28T04:05:50.733ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
15092024-08-28T04:05:50.733ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
15102024-08-28T04:05:50.733ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
15112024-08-28T04:05:50.733ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
15122024-08-28T04:05:50.733ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 793 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:61872 req_id = 328ab99e-617d-4ab6-9fee-0616fb99c6a0 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
15132024-08-28T04:05:50.930ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:38267
15142024-08-28T04:05:50.931ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 4b747939-586b-48cb-a488-7f7f972db309, kind: Service { id: 4eb364e4-14a2-46c4-b173-eb2890920e8f }, name: Name("external-dns-4eb364e4-14a2-46c4-b173-eb2890920e8f"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 246, 113])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a3e0f17f-d79d-4cc0-ba0f-997e844ad5f3, kind: Service { id: 94b35d7a-af71-4078-a57c-874ff9a27068 }, name: Name("external-dns-94b35d7a-af71-4078-a57c-874ff9a27068"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 165, 179])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0cddb53a-efd3-4f89-93d7-81b182af0cdb, kind: Service { id: 1c6ee351-cab6-4c23-8658-c31bac40b40a }, name: Name("nexus-1c6ee351-cab6-4c23-8658-c31bac40b40a"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 208, 245])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 1a83b6d1-627c-41ca-8268-a29f26800e91, kind: Service { id: 4c8a8ded-ced6-466f-8626-7fd39c37d020 }, name: Name("nexus-4c8a8ded-ced6-466f-8626-7fd39c37d020"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 220])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 34a49c6e-26fe-4abd-8702-695d279dbdca, kind: Service { id: bda2ed8e-2ea5-41eb-824b-d14527b76b8b }, name: Name("nexus-bda2ed8e-2ea5-41eb-824b-d14527b76b8b"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 210, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
15152024-08-28T04:05:50.931ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
15162024-08-28T04:05:50.931ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
15172024-08-28T04:05:50.931ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
15182024-08-28T04:05:50.931ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
15192024-08-28T04:05:50.931ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
15202024-08-28T04:05:50.931ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
15212024-08-28T04:05:50.931ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 358 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:38267 req_id = fe5afd96-b671-4201-833a-5cfd9b9d047c response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
15222024-08-28T04:05:55.987ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:51271
15232024-08-28T04:05:55.988ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1165 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:51271 req_id = 0271b611-e920-4e4b-a700-e965b211f31e response_code = 200 uri = /vpc-routes
15242024-08-28T04:05:56.060ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2147 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:51271 req_id = 6b40fc0d-db99-4002-957c-0cd965a65540 response_code = 204 uri = /vpc-routes
15252024-08-28T04:06:06.532ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:47021
15262024-08-28T04:06:06.532ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 58 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:47021 req_id = 2edce743-07cd-4c74-900a-f8b02317a401 response_code = 200 uri = /vpc-routes
15272024-08-28T04:06:06.541ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 105 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:47021 req_id = 3a64ea77-d3a0-46c7-92ab-fc433ea2b6a6 response_code = 204 uri = /vpc-routes
15282024-08-28T04:06:06.614ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:49337
15292024-08-28T04:06:06.615ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 410 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:49337 req_id = e84b6457-bcc0-4d71-bb17-1e58b98e87cb response_code = 200 uri = /v2p
15302024-08-28T04:06:06.693ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:47538
15312024-08-28T04:06:06.694ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
15322024-08-28T04:06:06.775ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
15332024-08-28T04:06:06.841ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 148339 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:47538 req_id = 6371d233-2542-4066-bf8e-4fe9eea79f2d response_code = 204 uri = /switch-ports
15342024-08-28T04:06:06.844ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2404 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:47538 req_id = e31aad04-23aa-4a86-a862-e6bcdc2ee557 response_code = 200 uri = /network-bootstore-config
15352024-08-28T04:06:06.907ZINFOSledAgent (bootstore): Attempting to update network config with generation: 2, current_generation: 1 file = bootstore/src/schemes/v0/peer.rs:564 peer_id = pc-sock-i86pc
15362024-08-28T04:06:06.907ZINFOSledAgent (bootstore): Writing ledger to /pool/int/51625293-e5a8-4cc4-9640-b64176e7a48e/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
15372024-08-28T04:06:06.909ZINFOSledAgent (bootstore): Writing ledger to /pool/int/4dad4c76-eb6c-4281-9f1d-a63fdb0117e1/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
15382024-08-28T04:06:06.910ZINFOSledAgent (bootstore): Broadcasting network config with generation 2 file = bootstore/src/schemes/v0/peer.rs:648 peer_id = pc-sock-i86pc
15392024-08-28T04:06:06.910ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 4614 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:47538 req_id = c41b2f23-9322-471d-8c95-ff4db525c864 response_code = 204 uri = /network-bootstore-config
15402024-08-28T04:06:07.249ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:52304
15412024-08-28T04:06:07.249ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 62 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:52304 req_id = bdd282a7-c08c-4992-ab92-7ca72e0370ef response_code = 200 uri = /vpc-routes
15422024-08-28T04:06:07.264ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:39900
15432024-08-28T04:06:07.264ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 869 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:39900 req_id = 71b6f0f6-ba4e-4eec-95cb-b0d7a19265bc response_code = 200 uri = /v2p
15442024-08-28T04:06:07.273ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 97 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:52304 req_id = ce1073fd-01d9-4cf4-8046-68a2873b3fca response_code = 204 uri = /vpc-routes
15452024-08-28T04:06:07.345ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:59899
15462024-08-28T04:06:07.346ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 133 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:59899 req_id = 5f8cad8d-a52f-4f46-a32b-bda716be32c9 response_code = 200 uri = /v2p
15472024-08-28T04:06:07.403ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:41774
15482024-08-28T04:06:07.403ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
15492024-08-28T04:06:07.441ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
15502024-08-28T04:06:07.498ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 95203 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:41774 req_id = 37eaec9f-8f69-482d-bc12-785808baac23 response_code = 204 uri = /switch-ports
15512024-08-28T04:06:07.498ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:32869
15522024-08-28T04:06:07.499ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 643 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:32869 req_id = 6359b832-a5b0-46dd-9b93-f27827c69890 response_code = 200 uri = /vpc-routes
15532024-08-28T04:06:07.499ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 227 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:41774 req_id = 4b2acd03-c4b3-49ae-818e-24ee12e63b48 response_code = 200 uri = /network-bootstore-config
15542024-08-28T04:06:07.508ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 128 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:32869 req_id = 68d4126b-809c-470c-af45-e65456956429 response_code = 204 uri = /vpc-routes
15552024-08-28T04:06:07.551ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:43936
15562024-08-28T04:06:07.552ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
15572024-08-28T04:06:07.590ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
15582024-08-28T04:06:07.644ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 93371 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:43936 req_id = 39733880-125f-4af1-9b56-dd89e085d709 response_code = 204 uri = /switch-ports
15592024-08-28T04:06:07.645ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 130 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:43936 req_id = 235169ce-da69-44f2-a2d7-5a9edf599589 response_code = 200 uri = /network-bootstore-config
15602024-08-28T04:06:08.958ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:33308
15612024-08-28T04:06:08.958ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 36 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:33308 req_id = e0a41183-1171-469c-beb0-bf1a5e7bdebb response_code = 200 uri = /baseboard
15622024-08-28T04:06:36.547ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:41613
15632024-08-28T04:06:36.547ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 62 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:41613 req_id = e24befae-711c-49a3-a7e3-2a0a3358eb75 response_code = 200 uri = /vpc-routes
15642024-08-28T04:06:36.560ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:55995
15652024-08-28T04:06:36.562ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 994 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:55995 req_id = 95fc6d25-8550-4075-b6b8-a2bde33201b3 response_code = 200 uri = /v2p
15662024-08-28T04:06:36.570ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 90 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:41613 req_id = 316305b9-9dc0-445e-a82e-179f87f0cb4f response_code = 204 uri = /vpc-routes
15672024-08-28T04:06:37.021ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:34642
15682024-08-28T04:06:37.022ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
15692024-08-28T04:06:37.073ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
15702024-08-28T04:06:37.176ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 155054 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:34642 req_id = 42109db2-8536-4f53-93c2-21f78bd7b244 response_code = 204 uri = /switch-ports
15712024-08-28T04:06:37.177ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 146 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:34642 req_id = dff35162-685f-4770-aaf6-38af5fd2764c response_code = 200 uri = /network-bootstore-config
15722024-08-28T04:06:37.199ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:63420
15732024-08-28T04:06:37.199ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 107 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:63420 req_id = 06739e32-3448-444e-857d-96849872cc58 response_code = 200 uri = /vpc-routes
15742024-08-28T04:06:37.215ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 164 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:63420 req_id = 411c10c2-4a63-4333-ac53-f3a396ee60b1 response_code = 204 uri = /vpc-routes
15752024-08-28T04:06:37.428ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:46327
15762024-08-28T04:06:37.428ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 64 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:46327 req_id = 31ae9a69-749a-4081-886d-db8404d7913c response_code = 200 uri = /vpc-routes
15772024-08-28T04:06:37.448ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 169 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:46327 req_id = 151909f2-3239-42e0-85b7-2596a7488d50 response_code = 204 uri = /vpc-routes
15782024-08-28T04:06:37.464ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:56297
15792024-08-28T04:06:37.464ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 137 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:56297 req_id = 49c5581d-4b7f-4345-b1d5-52b20dac586b response_code = 200 uri = /v2p
15802024-08-28T04:06:37.589ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:36813
15812024-08-28T04:06:37.589ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
15822024-08-28T04:06:37.629ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
15832024-08-28T04:06:37.686ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 97227 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:36813 req_id = a6de8715-99b8-43e1-bfea-4df22e3f6a53 response_code = 204 uri = /switch-ports
15842024-08-28T04:06:37.686ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:40973
15852024-08-28T04:06:37.687ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 638 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:36813 req_id = 2ae279bd-37c0-450a-bdde-cc19d2498b0c response_code = 200 uri = /network-bootstore-config
15862024-08-28T04:06:37.687ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 629 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:40973 req_id = b1fcb9df-284f-4924-9d91-ea8bdaeb8fc2 response_code = 200 uri = /v2p
15872024-08-28T04:06:37.687ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:50324
15882024-08-28T04:06:37.687ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
15892024-08-28T04:06:37.722ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
15902024-08-28T04:06:37.777ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 90312 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:50324 req_id = 5d5e9ac9-d8f5-4f69-b715-2aeef45d28c6 response_code = 204 uri = /switch-ports
15912024-08-28T04:06:37.778ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 117 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:50324 req_id = 9a35b77c-ec33-4d53-a739-4cfbe675b4b5 response_code = 200 uri = /network-bootstore-config
15922024-08-28T04:06:38.991ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:62779
15932024-08-28T04:06:38.992ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 721 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:62779 req_id = 3b58d22e-05b7-41e8-810c-fc8c1e8efb2b response_code = 200 uri = /baseboard
15942024-08-28T04:07:06.505ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:46445
15952024-08-28T04:07:06.505ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 76 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:46445 req_id = b28694a0-2cc3-4063-b98f-ecce8ec7307b response_code = 200 uri = /vpc-routes
15962024-08-28T04:07:06.515ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 120 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:46445 req_id = dd50c06f-f58b-4f95-a1e2-9a815ab75bb0 response_code = 204 uri = /vpc-routes
15972024-08-28T04:07:06.601ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:44294
15982024-08-28T04:07:06.601ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
15992024-08-28T04:07:06.638ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
16002024-08-28T04:07:06.693ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 91586 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:44294 req_id = 4fca9c39-c130-49db-b8ee-db67cb2de6bb response_code = 204 uri = /switch-ports
16012024-08-28T04:07:06.693ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:49670
16022024-08-28T04:07:06.693ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 156 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:44294 req_id = b2eb9463-9590-49d8-877c-f6ac5ec90362 response_code = 200 uri = /network-bootstore-config
16032024-08-28T04:07:06.693ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 166 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:49670 req_id = 0e59bc49-92c1-4dd9-94f1-ce9f5b524e2e response_code = 200 uri = /v2p
16042024-08-28T04:07:07.278ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:60594
16052024-08-28T04:07:07.278ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 82 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:60594 req_id = daac89d3-b9d3-445e-bf3f-63913f19a91f response_code = 200 uri = /vpc-routes
16062024-08-28T04:07:07.287ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 123 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:60594 req_id = 5f074b15-46b9-4049-98cf-5db989cac296 response_code = 204 uri = /vpc-routes
16072024-08-28T04:07:07.326ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:33960
16082024-08-28T04:07:07.326ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 117 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:33960 req_id = 8b458e53-5cd7-462f-8c12-34920bef5953 response_code = 200 uri = /v2p
16092024-08-28T04:07:07.333ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:35659
16102024-08-28T04:07:07.333ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
16112024-08-28T04:07:07.372ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
16122024-08-28T04:07:07.428ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 95094 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:35659 req_id = 5de1d2aa-83a6-4a54-a6b4-099d3502fab6 response_code = 204 uri = /switch-ports
16132024-08-28T04:07:07.429ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:45100
16142024-08-28T04:07:07.429ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:63422
16152024-08-28T04:07:07.429ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 89 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:45100 req_id = 8f8f939e-e481-40d7-8585-25d277540c66 response_code = 200 uri = /vpc-routes
16162024-08-28T04:07:07.429ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 152 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:63422 req_id = 7042f86c-ccfc-4795-8705-56da453efc27 response_code = 200 uri = /v2p
16172024-08-28T04:07:07.429ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 190 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:35659 req_id = 3888b445-ce2d-4fe1-bd5b-49d1227f90ae response_code = 200 uri = /network-bootstore-config
16182024-08-28T04:07:07.434ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:64278
16192024-08-28T04:07:07.434ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
16202024-08-28T04:07:07.473ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
16212024-08-28T04:07:07.532ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 97437 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:64278 req_id = 8cd2988c-8e30-4a6d-a97d-8fb035def6b9 response_code = 204 uri = /switch-ports
16222024-08-28T04:07:07.532ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 133 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:45100 req_id = 75ec4fba-0676-4758-8982-d986f320c322 response_code = 204 uri = /vpc-routes
16232024-08-28T04:07:07.532ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 173 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:64278 req_id = 63098386-7e2f-4fee-9996-726f8ecc8b12 response_code = 200 uri = /network-bootstore-config
16242024-08-28T04:07:09.026ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:62580
16252024-08-28T04:07:09.027ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 474 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:62580 req_id = d1e07617-4101-45b7-a723-f1da5fef7e96 response_code = 200 uri = /baseboard
16262024-08-28T04:07:36.619ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51857
16272024-08-28T04:07:36.619ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 70 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:51857 req_id = 794ed1f8-bb65-4bf0-ac21-6fdcd5d1be6b response_code = 200 uri = /vpc-routes
16282024-08-28T04:07:36.628ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 141 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:51857 req_id = 36a5ff43-0671-4f15-b51c-03101a1085fe response_code = 204 uri = /vpc-routes
16292024-08-28T04:07:36.698ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:55119
16302024-08-28T04:07:36.698ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 118 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:55119 req_id = 43930a57-c666-4113-aa8f-9554dfc77601 response_code = 200 uri = /v2p
16312024-08-28T04:07:36.845ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:52428
16322024-08-28T04:07:36.845ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
16332024-08-28T04:07:36.884ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
16342024-08-28T04:07:36.940ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 94528 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:52428 req_id = f4c2f8ac-2f6a-4ec3-bd00-7f3c655f3ff6 response_code = 204 uri = /switch-ports
16352024-08-28T04:07:36.940ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 115 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:52428 req_id = 87e20c4a-5a6b-4833-b22e-53a6a0d3de5b response_code = 200 uri = /network-bootstore-config
16362024-08-28T04:07:37.285ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:63715
16372024-08-28T04:07:37.285ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 83 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:63715 req_id = 6680f8ef-1d63-4d63-ac04-89871b8954c8 response_code = 200 uri = /vpc-routes
16382024-08-28T04:07:37.305ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1925 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:63715 req_id = 75bfb99f-7a7c-4a3c-92e7-d1c631f06ee4 response_code = 204 uri = /vpc-routes
16392024-08-28T04:07:37.545ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:51318
16402024-08-28T04:07:37.545ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 63 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:51318 req_id = 26a3fe4c-bef4-4c3a-b560-f9b1e594e0ce response_code = 200 uri = /vpc-routes
16412024-08-28T04:07:37.563ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 179 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:51318 req_id = 53f99d22-46ff-42bd-8d35-0d5d6ec79765 response_code = 204 uri = /vpc-routes
16422024-08-28T04:07:37.592ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:45963
16432024-08-28T04:07:37.593ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 133 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:45963 req_id = 4cb9fac1-4dc2-4b16-98d9-6de349b6becf response_code = 200 uri = /v2p
16442024-08-28T04:07:37.667ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:53624
16452024-08-28T04:07:37.667ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
16462024-08-28T04:07:37.707ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
16472024-08-28T04:07:37.761ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 93318 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:53624 req_id = de602c6a-33cd-490a-a47d-fd1ac02922b6 response_code = 204 uri = /switch-ports
16482024-08-28T04:07:37.761ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:57116
16492024-08-28T04:07:37.761ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:57399
16502024-08-28T04:07:37.761ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
16512024-08-28T04:07:37.765ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3569 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:53624 req_id = e63a6d79-3e35-45a4-b92a-8e6079de0380 response_code = 200 uri = /network-bootstore-config
16522024-08-28T04:07:37.765ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3590 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:57399 req_id = f8472f52-5dca-47d0-8ef3-7c70153693d9 response_code = 200 uri = /v2p
16532024-08-28T04:07:37.796ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
16542024-08-28T04:07:37.850ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 89409 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:57116 req_id = ae318304-da06-4730-abf5-9c6bc7215088 response_code = 204 uri = /switch-ports
16552024-08-28T04:07:37.851ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 108 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:57116 req_id = a49cc0ca-bc86-48d1-a0f7-febe79be19da response_code = 200 uri = /network-bootstore-config
16562024-08-28T04:07:39.061ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:33984
16572024-08-28T04:07:39.061ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 38 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:33984 req_id = 87b8d74c-8b22-4811-b4b4-11355492a37e response_code = 200 uri = /baseboard
16582024-08-28T04:08:06.521ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:57618
16592024-08-28T04:08:06.521ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 53 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:57618 req_id = 6fae2b3e-1bf3-446e-93d2-4a1ab83fc242 response_code = 200 uri = /vpc-routes
16602024-08-28T04:08:06.529ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 104 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:57618 req_id = ddfda081-9e56-4cd7-bc71-b895c8f3229d response_code = 204 uri = /vpc-routes
16612024-08-28T04:08:06.616ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:36142
16622024-08-28T04:08:06.616ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
16632024-08-28T04:08:06.656ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
16642024-08-28T04:08:06.711ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 94370 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:36142 req_id = 87033802-3cef-4d1c-82b9-81cdd3a3b559 response_code = 204 uri = /switch-ports
16652024-08-28T04:08:06.711ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:59247
16662024-08-28T04:08:06.711ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 130 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:59247 req_id = e0f5b98e-5d5d-46d3-ac2a-424eeb726320 response_code = 200 uri = /v2p
16672024-08-28T04:08:06.711ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 90 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:36142 req_id = 3e60c514-20d3-4f02-bf9d-a4ca1af6a979 response_code = 200 uri = /network-bootstore-config
16682024-08-28T04:08:07.179ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:56933
16692024-08-28T04:08:07.179ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 83 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:56933 req_id = 9196c9e0-18aa-4ddf-a9b0-e16a6d1b7868 response_code = 200 uri = /vpc-routes
16702024-08-28T04:08:07.195ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1971 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:56933 req_id = 9b89ed47-40af-49df-8d32-b60e0e3adb60 response_code = 204 uri = /vpc-routes
16712024-08-28T04:08:07.218ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:54874
16722024-08-28T04:08:07.218ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 48 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:54874 req_id = a2747805-70e9-4826-abdf-dbda306b1eb2 response_code = 200 uri = /vpc-routes
16732024-08-28T04:08:07.230ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 84 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:54874 req_id = f21dd5b4-d6a7-4728-bb67-e12eadced871 response_code = 204 uri = /vpc-routes
16742024-08-28T04:08:07.351ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:46179
16752024-08-28T04:08:07.351ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 163 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:46179 req_id = 3b0729d1-fa3a-4c9a-843f-0f4075be91ef response_code = 200 uri = /v2p
16762024-08-28T04:08:07.385ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:55331
16772024-08-28T04:08:07.385ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 90 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:55331 req_id = a1abdb3b-f742-415c-b919-961a34da5f0d response_code = 200 uri = /v2p
16782024-08-28T04:08:07.408ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:34711
16792024-08-28T04:08:07.409ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
16802024-08-28T04:08:07.449ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
16812024-08-28T04:08:07.507ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 98709 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:34711 req_id = 8cce1131-ddb3-40d6-8826-b3b9673189a7 response_code = 204 uri = /switch-ports
16822024-08-28T04:08:07.507ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:56349
16832024-08-28T04:08:07.507ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
16842024-08-28T04:08:07.542ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
16852024-08-28T04:08:07.597ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 89792 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:56349 req_id = 3b0d10e3-58c8-4c0d-b961-0bbf886ae7b1 response_code = 204 uri = /switch-ports
16862024-08-28T04:08:07.597ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 141 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:34711 req_id = 44bfb029-f6ea-4892-8f15-c512ced5d665 response_code = 200 uri = /network-bootstore-config
16872024-08-28T04:08:07.598ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 693 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:56349 req_id = 60a839a4-a8f9-4d0a-b0fd-79052c8e1603 response_code = 200 uri = /network-bootstore-config
16882024-08-28T04:08:09.093ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:37598
16892024-08-28T04:08:09.093ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 51 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:37598 req_id = 9ab320de-65ef-48be-b3e2-9ccb78adb086 response_code = 200 uri = /baseboard
16902024-08-28T04:08:16.896ZINFOSledAgent (dropshot): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:60627 remote_addr = [fd00:1122:3344:101::d]:48245 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f
16912024-08-28T04:08:17.034ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 136957 local_addr = [fd00:1122:3344:101::1]:60627 method = GET remote_addr = [fd00:1122:3344:101::d]:48245 req_id = f5c9f4ff-72e5-43de-9db1-d77ec6f4b31b response_code = 200 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f uri = /f62e5a13-51f3-4379-acf3-2f8e8601ac1f
16922024-08-28T04:08:36.624ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:63232
16932024-08-28T04:08:36.624ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 74 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:63232 req_id = e1b30197-52f6-4c4c-a010-043cbe653d20 response_code = 200 uri = /vpc-routes
16942024-08-28T04:08:36.634ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 123 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:63232 req_id = fd07050b-f308-422f-9356-c0cda1a79554 response_code = 204 uri = /vpc-routes
16952024-08-28T04:08:36.753ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:54387
16962024-08-28T04:08:36.753ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 94 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:54387 req_id = 71c653b4-6bf1-4e91-accb-a8091bf80aaa response_code = 200 uri = /v2p
16972024-08-28T04:08:36.891ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:43468
16982024-08-28T04:08:36.891ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
16992024-08-28T04:08:36.932ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
17002024-08-28T04:08:36.992ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 101032 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:43468 req_id = 1b0a9f60-dfad-4848-8bd8-d435d0c2bdec response_code = 204 uri = /switch-ports
17012024-08-28T04:08:36.993ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 107 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:43468 req_id = 45a95e70-9562-4dbd-9db9-ae8a33fdc8ab response_code = 200 uri = /network-bootstore-config
17022024-08-28T04:08:37.303ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:59861
17032024-08-28T04:08:37.303ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 88 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:59861 req_id = 8dba3dd8-172d-4922-bc3d-389ad76953ff response_code = 200 uri = /vpc-routes
17042024-08-28T04:08:37.316ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 160 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:59861 req_id = c1ea1f61-377c-47ff-8a5d-642822a939aa response_code = 204 uri = /vpc-routes
17052024-08-28T04:08:37.381ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:33155
17062024-08-28T04:08:37.381ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 166 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:33155 req_id = fa6378dc-f914-4f71-8496-8885b7ff2aba response_code = 200 uri = /v2p
17072024-08-28T04:08:37.457ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:49243
17082024-08-28T04:08:37.457ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 80 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:49243 req_id = 5c1bd62b-d9c6-47c9-8a70-0b330ecb0031 response_code = 200 uri = /vpc-routes
17092024-08-28T04:08:37.473ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1780 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:49243 req_id = d24e96dc-f99d-43b6-99c0-2a11b33e84dd response_code = 204 uri = /vpc-routes
17102024-08-28T04:08:37.519ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:58507
17112024-08-28T04:08:37.519ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 139 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:58507 req_id = b53c6999-ce6e-492c-bc85-108335a13958 response_code = 200 uri = /v2p
17122024-08-28T04:08:37.597ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:49505
17132024-08-28T04:08:37.597ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
17142024-08-28T04:08:37.639ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
17152024-08-28T04:08:37.698ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 101062 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:49505 req_id = c0877258-1d44-4a9a-9b71-b50c1a3b9a78 response_code = 204 uri = /switch-ports
17162024-08-28T04:08:37.698ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:44700
17172024-08-28T04:08:37.698ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
17182024-08-28T04:08:37.702ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 4053 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:49505 req_id = caafc45a-5c6f-4a5e-b6e8-39062736a940 response_code = 200 uri = /network-bootstore-config
17192024-08-28T04:08:37.733ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
17202024-08-28T04:08:37.790ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 92460 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:44700 req_id = 54602cfb-d42d-4242-a9bc-9c0b8f0fa7b0 response_code = 204 uri = /switch-ports
17212024-08-28T04:08:37.791ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 815 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:44700 req_id = 4eec6345-d33b-49f0-9e29-d3d519690dd0 response_code = 200 uri = /network-bootstore-config
17222024-08-28T04:08:39.128ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:49170
17232024-08-28T04:08:39.128ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 39 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:49170 req_id = 500421c6-066e-4af0-8c26-32d4613e7d0e response_code = 200 uri = /baseboard
17242024-08-28T04:08:46.903ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8210 local_addr = [fd00:1122:3344:101::1]:60627 method = GET remote_addr = [fd00:1122:3344:101::d]:48245 req_id = e2ceb717-4dfc-4524-b022-990fc107406d response_code = 200 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f uri = /f62e5a13-51f3-4379-acf3-2f8e8601ac1f
17252024-08-28T04:09:06.523ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:55475
17262024-08-28T04:09:06.523ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 83 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:55475 req_id = 7393db40-0e6e-46f5-b271-fff209674f98 response_code = 200 uri = /vpc-routes
17272024-08-28T04:09:06.626ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:56550
17282024-08-28T04:09:06.626ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 154 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:56550 req_id = 2922d0d3-2abb-4ef3-8318-12134c7b1a0e response_code = 200 uri = /v2p
17292024-08-28T04:09:06.629ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 99 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:55475 req_id = a2758a99-061a-4361-ad0d-42a03a1c8b1e response_code = 204 uri = /vpc-routes
17302024-08-28T04:09:06.680ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:42106
17312024-08-28T04:09:06.680ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
17322024-08-28T04:09:06.720ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
17332024-08-28T04:09:06.786ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 105704 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:42106 req_id = 2f3ef6b8-5f99-4658-a611-cef484744ee0 response_code = 204 uri = /switch-ports
17342024-08-28T04:09:06.786ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 98 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:42106 req_id = 602e1ce4-a342-4d68-bc80-5a07e10bfe8c response_code = 200 uri = /network-bootstore-config
17352024-08-28T04:09:07.303ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:53227
17362024-08-28T04:09:07.303ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 106 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:53227 req_id = da28895d-5553-4656-9e94-776941201ea5 response_code = 200 uri = /vpc-routes
17372024-08-28T04:09:07.312ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 143 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:53227 req_id = b6a0dc61-a661-45ca-a40f-6bfc997fbe28 response_code = 204 uri = /vpc-routes
17382024-08-28T04:09:07.358ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:46750
17392024-08-28T04:09:07.358ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 38 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:46750 req_id = eede5a34-38fd-4ce7-af8d-7617f81bc23a response_code = 200 uri = /vpc-routes
17402024-08-28T04:09:07.367ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 101 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:46750 req_id = 8cc18b94-5b50-4fc8-8266-63250cc14c8d response_code = 204 uri = /vpc-routes
17412024-08-28T04:09:07.393ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:37749
17422024-08-28T04:09:07.393ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
17432024-08-28T04:09:07.434ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
17442024-08-28T04:09:07.499ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 105897 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:37749 req_id = 5496166f-6af0-47e4-8b7a-3476474b15e8 response_code = 204 uri = /switch-ports
17452024-08-28T04:09:07.499ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:52914
17462024-08-28T04:09:07.499ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:51467
17472024-08-28T04:09:07.499ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:45486
17482024-08-28T04:09:07.499ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
17492024-08-28T04:09:07.504ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 4341 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:45486 req_id = 1e64bb66-ce71-4080-8d47-0806ab052897 response_code = 200 uri = /v2p
17502024-08-28T04:09:07.504ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 4872 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:51467 req_id = 0ca41aaa-c0f9-4c36-9e9b-0c178c734046 response_code = 200 uri = /v2p
17512024-08-28T04:09:07.504ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 179 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:37749 req_id = 678776cd-e799-4cd2-8b57-45a690a45309 response_code = 200 uri = /network-bootstore-config
17522024-08-28T04:09:07.542ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
17532024-08-28T04:09:07.605ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 105482 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:52914 req_id = cf082de0-f680-4097-b668-e5678d9c282c response_code = 204 uri = /switch-ports
17542024-08-28T04:09:07.605ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 680 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:52914 req_id = 092055c7-dce7-45a3-99fc-2751ae6279c7 response_code = 200 uri = /network-bootstore-config
17552024-08-28T04:09:09.162ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:51057
17562024-08-28T04:09:09.162ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 49 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:51057 req_id = 1734b04d-87bf-4225-80ca-c92b52cf83e1 response_code = 200 uri = /baseboard
17572024-08-28T04:09:16.901ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 6999 local_addr = [fd00:1122:3344:101::1]:60627 method = GET remote_addr = [fd00:1122:3344:101::d]:48245 req_id = 99dfe856-afae-4d80-95a6-bc0c36816f53 response_code = 200 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f uri = /f62e5a13-51f3-4379-acf3-2f8e8601ac1f
17582024-08-28T04:09:36.591ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:50501
17592024-08-28T04:09:36.591ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 64 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:50501 req_id = e08bd6ba-9ece-4c6a-a548-4b18e2bc9314 response_code = 200 uri = /vpc-routes
17602024-08-28T04:09:36.608ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1014 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:50501 req_id = 9c175536-b6d1-4fe8-bc31-aba49dddad61 response_code = 204 uri = /vpc-routes
17612024-08-28T04:09:37.032ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:41007
17622024-08-28T04:09:37.032ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 129 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:41007 req_id = a1a1ad77-82e6-4a14-9e90-0048f5fc2ef7 response_code = 200 uri = /v2p
17632024-08-28T04:09:37.205ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:48355
17642024-08-28T04:09:37.205ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
17652024-08-28T04:09:37.258ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
17662024-08-28T04:09:37.326ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 121527 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:48355 req_id = 685bd180-399a-4ad0-aae9-6e7d6b6c360f response_code = 204 uri = /switch-ports
17672024-08-28T04:09:37.327ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 147 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:48355 req_id = c2f4bc66-3514-4382-9038-5ff67d511a01 response_code = 200 uri = /network-bootstore-config
17682024-08-28T04:09:37.390ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:50331
17692024-08-28T04:09:37.390ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 79 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:50331 req_id = 1dc78a90-a105-4d52-9586-99511962819e response_code = 200 uri = /vpc-routes
17702024-08-28T04:09:37.400ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 146 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:50331 req_id = 2fb44be2-6188-4b8c-8bb1-548d3b7cbdb3 response_code = 204 uri = /vpc-routes
17712024-08-28T04:09:37.489ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:43196
17722024-08-28T04:09:37.489ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 58 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:43196 req_id = dc60ed65-6204-47d7-a9de-62d8e1c62c59 response_code = 200 uri = /vpc-routes
17732024-08-28T04:09:37.502ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 685 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:43196 req_id = 7d2f133c-cadf-43d2-be45-2128c51bac79 response_code = 204 uri = /vpc-routes
17742024-08-28T04:09:37.613ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:57490
17752024-08-28T04:09:37.613ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 159 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:57490 req_id = 9fa3c78e-6de3-4cfe-a9ea-f06ca23433fd response_code = 200 uri = /v2p
17762024-08-28T04:09:37.646ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:33066
17772024-08-28T04:09:37.646ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
17782024-08-28T04:09:37.692ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
17792024-08-28T04:09:37.772ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 125757 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:33066 req_id = 17ec162f-a194-482d-a028-76e25b7bb9b7 response_code = 204 uri = /switch-ports
17802024-08-28T04:09:37.772ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:41908
17812024-08-28T04:09:37.772ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:53437
17822024-08-28T04:09:37.772ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
17832024-08-28T04:09:37.778ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 5962 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:53437 req_id = 4ea858cf-e294-4df3-b97b-838412b5b61a response_code = 200 uri = /v2p
17842024-08-28T04:09:37.778ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 5996 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:33066 req_id = 8f03d307-ea87-41e2-90dd-d0523ab6b15d response_code = 200 uri = /network-bootstore-config
17852024-08-28T04:09:37.821ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
17862024-08-28T04:09:37.896ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 123871 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:41908 req_id = eed39350-de5d-4435-9c5b-1ae538865bcf response_code = 204 uri = /switch-ports
17872024-08-28T04:09:37.897ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 142 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:41908 req_id = c550b0c7-cf0c-455b-b326-82dee74e8672 response_code = 200 uri = /network-bootstore-config
17882024-08-28T04:09:39.194ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:65054
17892024-08-28T04:09:39.194ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 59 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:65054 req_id = 8d553eda-06fe-47ec-9076-8ca752470ab2 response_code = 200 uri = /baseboard
17902024-08-28T04:09:46.901ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 7203 local_addr = [fd00:1122:3344:101::1]:60627 method = GET remote_addr = [fd00:1122:3344:101::d]:48245 req_id = 5d789a90-1047-43fd-8462-b5424cc80018 response_code = 200 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f uri = /f62e5a13-51f3-4379-acf3-2f8e8601ac1f
17912024-08-28T04:09:48.263ZINFOSledAgent (auto-cleanup-task): running automatic periodic zone bundle cleanup file = sled-agent/src/zone_bundle.rs:197
17922024-08-28T04:09:48.263ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
17932024-08-28T04:09:48.305ZINFOSledAgent (auto-cleanup-task): top of bundle cleanup loop file = sled-agent/src/zone_bundle.rs:186 next_cleanup = Instant { tv_sec: 1875, tv_nsec: 389423875 } time_to_next_cleanup = 599.99999799s
17942024-08-28T04:10:02.560ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
17952024-08-28T04:10:06.633ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:39422
17962024-08-28T04:10:06.633ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 63 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:39422 req_id = 05237ee5-4282-455c-b0d3-3338ce98a295 response_code = 200 uri = /vpc-routes
17972024-08-28T04:10:06.643ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 116 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:39422 req_id = dc5f25fa-af8d-4cb5-81e4-4edf2f66eb7f response_code = 204 uri = /vpc-routes
17982024-08-28T04:10:06.741ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51641
17992024-08-28T04:10:06.742ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 708 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:51641 req_id = 6e97c1b7-ef4b-4ea0-8e1c-0a64bb426cae response_code = 200 uri = /v2p
18002024-08-28T04:10:06.755ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:33283
18012024-08-28T04:10:06.755ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
18022024-08-28T04:10:06.797ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
18032024-08-28T04:10:06.857ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 101801 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:33283 req_id = 95127be5-c086-446e-af66-75550f340164 response_code = 204 uri = /switch-ports
18042024-08-28T04:10:06.857ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 94 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:33283 req_id = 6617f991-e4ef-414c-99b3-5fd74561977c response_code = 200 uri = /network-bootstore-config
18052024-08-28T04:10:07.306ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:34913
18062024-08-28T04:10:07.307ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 98 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:34913 req_id = 40d59be2-0a28-4008-af8f-c1ac2d51d01a response_code = 200 uri = /vpc-routes
18072024-08-28T04:10:07.327ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1110 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:34913 req_id = a055d036-9b87-4196-9093-9183240715da response_code = 204 uri = /vpc-routes
18082024-08-28T04:10:07.338ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:53073
18092024-08-28T04:10:07.338ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 43 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:53073 req_id = 61d0b1dd-ae39-4d6d-bf4d-76a9a6b484a9 response_code = 200 uri = /vpc-routes
18102024-08-28T04:10:07.349ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:34109
18112024-08-28T04:10:07.349ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 115 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:34109 req_id = 989f9e61-c66d-4bc2-a6a5-bc473b1a10e3 response_code = 200 uri = /v2p
18122024-08-28T04:10:07.358ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 85 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:53073 req_id = 4fd2604e-6189-45cd-ae04-18584e558ad7 response_code = 204 uri = /vpc-routes
18132024-08-28T04:10:07.366ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:41278
18142024-08-28T04:10:07.366ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
18152024-08-28T04:10:07.413ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
18162024-08-28T04:10:07.488ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 121815 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:41278 req_id = 6699b0cf-8bcc-4d29-976c-67cf7b8acc71 response_code = 204 uri = /switch-ports
18172024-08-28T04:10:07.488ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:50268
18182024-08-28T04:10:07.488ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
18192024-08-28T04:10:07.492ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 4110 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:41278 req_id = d81a78d9-0dcb-4f81-9ad6-ad03fe8c1058 response_code = 200 uri = /network-bootstore-config
18202024-08-28T04:10:07.528ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
18212024-08-28T04:10:07.532ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:61368
18222024-08-28T04:10:07.533ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 231 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:61368 req_id = fbd9e324-7f53-480c-aa08-587da7e58569 response_code = 200 uri = /v2p
18232024-08-28T04:10:07.590ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 102612 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:50268 req_id = f5965dca-9d0f-4dee-9b84-04965275b7df response_code = 204 uri = /switch-ports
18242024-08-28T04:10:07.591ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 135 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:50268 req_id = a7d2aec1-5e29-482c-aa4c-63f14adf120d response_code = 200 uri = /network-bootstore-config
18252024-08-28T04:10:09.228ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:39252
18262024-08-28T04:10:09.228ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 42 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:39252 req_id = afeda9fe-efb9-4a1f-ba87-9b3277d9226d response_code = 200 uri = /baseboard
18272024-08-28T04:10:16.914ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8601 local_addr = [fd00:1122:3344:101::1]:60627 method = GET remote_addr = [fd00:1122:3344:101::d]:48245 req_id = ce1db346-0fed-40d6-a8e7-f44a75699a7a response_code = 200 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f uri = /f62e5a13-51f3-4379-acf3-2f8e8601ac1f
18282024-08-28T04:10:36.584ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:38217
18292024-08-28T04:10:36.584ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 65 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:38217 req_id = dc5d1c82-7b72-4b3b-991f-00710cafe473 response_code = 200 uri = /vpc-routes
18302024-08-28T04:10:36.601ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1562 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:38217 req_id = e4efb7b2-56e7-4a87-b5b0-108ec5c6967f response_code = 204 uri = /vpc-routes
18312024-08-28T04:10:36.823ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:54833
18322024-08-28T04:10:36.823ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 164 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:54833 req_id = 4c6f6024-869c-4e76-8ee1-d2eef272eb13 response_code = 200 uri = /v2p
18332024-08-28T04:10:36.976ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:48333
18342024-08-28T04:10:36.976ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
18352024-08-28T04:10:37.018ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
18362024-08-28T04:10:37.135ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 159209 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:48333 req_id = 39c3f4a5-b275-4c08-b665-3fed000986c4 response_code = 204 uri = /switch-ports
18372024-08-28T04:10:37.135ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:60700
18382024-08-28T04:10:37.135ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 4b747939-586b-48cb-a488-7f7f972db309, kind: Service { id: 4eb364e4-14a2-46c4-b173-eb2890920e8f }, name: Name("external-dns-4eb364e4-14a2-46c4-b173-eb2890920e8f"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 246, 113])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a3e0f17f-d79d-4cc0-ba0f-997e844ad5f3, kind: Service { id: 94b35d7a-af71-4078-a57c-874ff9a27068 }, name: Name("external-dns-94b35d7a-af71-4078-a57c-874ff9a27068"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 165, 179])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0cddb53a-efd3-4f89-93d7-81b182af0cdb, kind: Service { id: 1c6ee351-cab6-4c23-8658-c31bac40b40a }, name: Name("nexus-1c6ee351-cab6-4c23-8658-c31bac40b40a"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 208, 245])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 1a83b6d1-627c-41ca-8268-a29f26800e91, kind: Service { id: 4c8a8ded-ced6-466f-8626-7fd39c37d020 }, name: Name("nexus-4c8a8ded-ced6-466f-8626-7fd39c37d020"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 220])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 34a49c6e-26fe-4abd-8702-695d279dbdca, kind: Service { id: bda2ed8e-2ea5-41eb-824b-d14527b76b8b }, name: Name("nexus-bda2ed8e-2ea5-41eb-824b-d14527b76b8b"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 210, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
18392024-08-28T04:10:37.136ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18402024-08-28T04:10:37.136ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18412024-08-28T04:10:37.136ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18422024-08-28T04:10:37.136ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
18432024-08-28T04:10:37.136ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
18442024-08-28T04:10:37.136ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 434 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:48333 req_id = e31f3b7f-91d9-4d5d-9f5a-c58893e34d0f response_code = 200 uri = /network-bootstore-config
18452024-08-28T04:10:37.136ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
18462024-08-28T04:10:37.136ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 610 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:60700 req_id = 47e4d09c-4d65-4881-93b5-6cbcf7ee2ede response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
18472024-08-28T04:10:37.370ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:53896
18482024-08-28T04:10:37.370ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 115 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:53896 req_id = aa402fcf-1f3b-4c0d-a48a-4db7c37e4327 response_code = 200 uri = /vpc-routes
18492024-08-28T04:10:37.387ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 189 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:53896 req_id = 81a23018-e79e-440c-8d9e-0ee448e56cc2 response_code = 204 uri = /vpc-routes
18502024-08-28T04:10:37.461ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:48869
18512024-08-28T04:10:37.461ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 65 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:48869 req_id = df4f42bb-e202-41f1-a599-eb4d441c40db response_code = 200 uri = /vpc-routes
18522024-08-28T04:10:37.473ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 121 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:48869 req_id = dd0dc3ba-11b2-4a5d-9a3a-95fba156f9ea response_code = 204 uri = /vpc-routes
18532024-08-28T04:10:37.522ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:57494
18542024-08-28T04:10:37.523ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1030 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:57494 req_id = 8a1e0905-f675-4a05-a54b-c410b945f13f response_code = 200 uri = /v2p
18552024-08-28T04:10:37.538ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:52506
18562024-08-28T04:10:37.538ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
18572024-08-28T04:10:37.586ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
18582024-08-28T04:10:37.661ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 123007 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:52506 req_id = 4014118e-95a2-4362-8da7-7ba9c4c526a0 response_code = 204 uri = /switch-ports
18592024-08-28T04:10:37.661ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:53240
18602024-08-28T04:10:37.661ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:42532
18612024-08-28T04:10:37.662ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 332 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:52506 req_id = a2f3c3b5-ae28-4f9e-8ddc-c62745b6e90c response_code = 200 uri = /network-bootstore-config
18622024-08-28T04:10:37.662ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
18632024-08-28T04:10:37.662ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 873 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:53240 req_id = 93df73c1-4bd5-4387-8f87-9b14be2d544b response_code = 200 uri = /v2p
18642024-08-28T04:10:37.688ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:55671
18652024-08-28T04:10:37.688ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 4b747939-586b-48cb-a488-7f7f972db309, kind: Service { id: 4eb364e4-14a2-46c4-b173-eb2890920e8f }, name: Name("external-dns-4eb364e4-14a2-46c4-b173-eb2890920e8f"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 246, 113])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a3e0f17f-d79d-4cc0-ba0f-997e844ad5f3, kind: Service { id: 94b35d7a-af71-4078-a57c-874ff9a27068 }, name: Name("external-dns-94b35d7a-af71-4078-a57c-874ff9a27068"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 165, 179])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0cddb53a-efd3-4f89-93d7-81b182af0cdb, kind: Service { id: 1c6ee351-cab6-4c23-8658-c31bac40b40a }, name: Name("nexus-1c6ee351-cab6-4c23-8658-c31bac40b40a"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 208, 245])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 1a83b6d1-627c-41ca-8268-a29f26800e91, kind: Service { id: 4c8a8ded-ced6-466f-8626-7fd39c37d020 }, name: Name("nexus-4c8a8ded-ced6-466f-8626-7fd39c37d020"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 220])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 34a49c6e-26fe-4abd-8702-695d279dbdca, kind: Service { id: bda2ed8e-2ea5-41eb-824b-d14527b76b8b }, name: Name("nexus-bda2ed8e-2ea5-41eb-824b-d14527b76b8b"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 210, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
18662024-08-28T04:10:37.688ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18672024-08-28T04:10:37.688ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18682024-08-28T04:10:37.688ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18692024-08-28T04:10:37.688ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
18702024-08-28T04:10:37.688ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
18712024-08-28T04:10:37.688ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
18722024-08-28T04:10:37.688ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 373 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:55671 req_id = 60f344d9-489f-4947-8226-b7cce8745162 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
18732024-08-28T04:10:37.703ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
18742024-08-28T04:10:37.711ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:61143
18752024-08-28T04:10:37.712ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 4b747939-586b-48cb-a488-7f7f972db309, kind: Service { id: 4eb364e4-14a2-46c4-b173-eb2890920e8f }, name: Name("external-dns-4eb364e4-14a2-46c4-b173-eb2890920e8f"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 246, 113])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a3e0f17f-d79d-4cc0-ba0f-997e844ad5f3, kind: Service { id: 94b35d7a-af71-4078-a57c-874ff9a27068 }, name: Name("external-dns-94b35d7a-af71-4078-a57c-874ff9a27068"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 165, 179])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0cddb53a-efd3-4f89-93d7-81b182af0cdb, kind: Service { id: 1c6ee351-cab6-4c23-8658-c31bac40b40a }, name: Name("nexus-1c6ee351-cab6-4c23-8658-c31bac40b40a"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 208, 245])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 1a83b6d1-627c-41ca-8268-a29f26800e91, kind: Service { id: 4c8a8ded-ced6-466f-8626-7fd39c37d020 }, name: Name("nexus-4c8a8ded-ced6-466f-8626-7fd39c37d020"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 220])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 34a49c6e-26fe-4abd-8702-695d279dbdca, kind: Service { id: bda2ed8e-2ea5-41eb-824b-d14527b76b8b }, name: Name("nexus-bda2ed8e-2ea5-41eb-824b-d14527b76b8b"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 210, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
18762024-08-28T04:10:37.712ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18772024-08-28T04:10:37.712ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18782024-08-28T04:10:37.712ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18792024-08-28T04:10:37.712ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
18802024-08-28T04:10:37.712ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
18812024-08-28T04:10:37.712ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
18822024-08-28T04:10:37.712ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 317 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:61143 req_id = e5d5a6d6-c2d2-4d67-8772-e0be1fb3adcd response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
18832024-08-28T04:10:37.768ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 106595 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:42532 req_id = f53bcedb-832d-4ae9-8adc-60fbe1a377f6 response_code = 204 uri = /switch-ports
18842024-08-28T04:10:37.768ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 152 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:42532 req_id = 91393d9c-8c9f-475a-ba77-f622520b8682 response_code = 200 uri = /network-bootstore-config
18852024-08-28T04:10:39.262ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:46055
18862024-08-28T04:10:39.262ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 66 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:46055 req_id = 8732cb5e-beaf-47c5-b756-584134199301 response_code = 200 uri = /baseboard
18872024-08-28T04:10:46.978ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 7241 local_addr = [fd00:1122:3344:101::1]:60627 method = GET remote_addr = [fd00:1122:3344:101::d]:48245 req_id = 730e4958-84b6-4cd2-bdf5-36c3bf9ea30d response_code = 200 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f uri = /f62e5a13-51f3-4379-acf3-2f8e8601ac1f
18882024-08-28T04:11:06.587ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:61096
18892024-08-28T04:11:06.587ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 77 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:61096 req_id = a8535465-6b94-4d74-98a0-770686a722e4 response_code = 200 uri = /vpc-routes
18902024-08-28T04:11:06.624ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:44134
18912024-08-28T04:11:06.624ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 131 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:44134 req_id = fda60a6e-4ff1-4b98-8c03-eba487e34029 response_code = 200 uri = /v2p
18922024-08-28T04:11:06.631ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 94 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:61096 req_id = 235797b5-772d-41c2-95bc-28289a32001d response_code = 204 uri = /vpc-routes
18932024-08-28T04:11:06.695ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:33636
18942024-08-28T04:11:06.695ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
18952024-08-28T04:11:06.738ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
18962024-08-28T04:11:06.798ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 102654 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:33636 req_id = 7ba4b042-2e34-4abe-8720-ef4281dfc750 response_code = 204 uri = /switch-ports
18972024-08-28T04:11:06.798ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 115 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:33636 req_id = 748b998b-708b-4d73-ac63-9867d1e65c38 response_code = 200 uri = /network-bootstore-config
18982024-08-28T04:11:07.352ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:35471
18992024-08-28T04:11:07.353ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 869 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:35471 req_id = 2c2877a7-71e8-4b8a-9c7c-3b74c777f0b7 response_code = 200 uri = /v2p
19002024-08-28T04:11:07.353ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:43400
19012024-08-28T04:11:07.354ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 55 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:43400 req_id = bbe6a5ab-a3e6-454d-9145-ca376f791a9b response_code = 200 uri = /vpc-routes
19022024-08-28T04:11:07.365ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 138 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:43400 req_id = c4838182-ac0f-44d9-beef-9a2ae787219c response_code = 204 uri = /vpc-routes
19032024-08-28T04:11:07.412ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:39461
19042024-08-28T04:11:07.412ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 40 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:39461 req_id = 2481d6f7-191f-4898-a373-5a7a3ee14403 response_code = 200 uri = /vpc-routes
19052024-08-28T04:11:07.423ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 689 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:39461 req_id = 9da7621c-feca-43d9-aee3-3f685ebbd06d response_code = 204 uri = /vpc-routes
19062024-08-28T04:11:07.446ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:44162
19072024-08-28T04:11:07.447ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 106 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:44162 req_id = ce89a4ec-3a9e-489e-931d-01be090b7b99 response_code = 200 uri = /v2p
19082024-08-28T04:11:07.459ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:59033
19092024-08-28T04:11:07.459ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
19102024-08-28T04:11:07.500ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
19112024-08-28T04:11:07.559ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 99454 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:59033 req_id = b2eab94b-21dd-4c10-b42b-6e767cd9026c response_code = 204 uri = /switch-ports
19122024-08-28T04:11:07.559ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:65110
19132024-08-28T04:11:07.559ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
19142024-08-28T04:11:07.595ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
19152024-08-28T04:11:07.655ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 96346 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:65110 req_id = 0df08b98-7e49-4983-a0c1-7d9eefb57b93 response_code = 204 uri = /switch-ports
19162024-08-28T04:11:07.656ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 113 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:59033 req_id = 0bfc16c5-0618-41d4-aa28-3eef12b4e06a response_code = 200 uri = /network-bootstore-config
19172024-08-28T04:11:07.656ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 758 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:65110 req_id = 6fe64949-64ee-460e-9868-40a1ada41794 response_code = 200 uri = /network-bootstore-config
19182024-08-28T04:11:09.296ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:47583
19192024-08-28T04:11:09.296ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 49 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:47583 req_id = b5f0e18f-8753-490b-8773-e17cb89b702f response_code = 200 uri = /baseboard
19202024-08-28T04:11:16.978ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 7498 local_addr = [fd00:1122:3344:101::1]:60627 method = GET remote_addr = [fd00:1122:3344:101::d]:48245 req_id = e757cb64-66e8-4695-ae2c-e2d2c6ac09ce response_code = 200 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f uri = /f62e5a13-51f3-4379-acf3-2f8e8601ac1f
19212024-08-28T04:11:36.517ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:38670
19222024-08-28T04:11:36.517ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 65 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:38670 req_id = 912407ec-0004-4c45-abda-1bc82a15fc1b response_code = 200 uri = /vpc-routes
19232024-08-28T04:11:36.535ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 149 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:38670 req_id = ca2f3ebe-6081-4299-bebe-3b4a77e17ffb response_code = 204 uri = /vpc-routes
19242024-08-28T04:11:36.704ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:45553
19252024-08-28T04:11:36.704ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
19262024-08-28T04:11:36.750ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
19272024-08-28T04:11:36.808ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 103938 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:45553 req_id = 6b1283b3-f843-4bc6-82d5-7440b6fc5dc1 response_code = 204 uri = /switch-ports
19282024-08-28T04:11:36.808ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:35691
19292024-08-28T04:11:36.809ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 166 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:35691 req_id = b3481d50-1bc8-4d2c-8a27-0676eee6f032 response_code = 200 uri = /v2p
19302024-08-28T04:11:36.809ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 101 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:45553 req_id = 02906793-c236-4166-ab0f-81c8fc3f39a6 response_code = 200 uri = /network-bootstore-config
19312024-08-28T04:11:37.314ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:40902
19322024-08-28T04:11:37.314ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 119 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:40902 req_id = 69b05cb4-dcf7-4f01-8978-07d9ca4725aa response_code = 200 uri = /vpc-routes
19332024-08-28T04:11:37.324ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 929 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:40902 req_id = d392d6e2-7191-461b-a9fc-95c8ea4c5c0e response_code = 204 uri = /vpc-routes
19342024-08-28T04:11:37.399ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:62692
19352024-08-28T04:11:37.399ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 119 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:62692 req_id = 243f57eb-ec63-4e73-8ed3-6465a45bf00c response_code = 200 uri = /v2p
19362024-08-28T04:11:37.474ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:58853
19372024-08-28T04:11:37.474ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 61 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:58853 req_id = cf0c0e5f-aad1-43e5-beae-4b6ce4975555 response_code = 200 uri = /vpc-routes
19382024-08-28T04:11:37.568ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:53148
19392024-08-28T04:11:37.568ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
19402024-08-28T04:11:37.612ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
19412024-08-28T04:11:37.691ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 122284 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:53148 req_id = d2a764e0-8cb5-400c-a583-8d04229371d3 response_code = 204 uri = /switch-ports
19422024-08-28T04:11:37.691ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:50960
19432024-08-28T04:11:37.691ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:36895
19442024-08-28T04:11:37.691ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 223 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:58853 req_id = dac1b145-1322-42d0-a832-ca184c0c43c0 response_code = 204 uri = /vpc-routes
19452024-08-28T04:11:37.691ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 231 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:53148 req_id = 5fbc02c8-7410-4fbb-92bc-95920e06b115 response_code = 200 uri = /network-bootstore-config
19462024-08-28T04:11:37.691ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 230 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:36895 req_id = f70f7654-d2fa-4857-9183-5203e8170d96 response_code = 200 uri = /v2p
19472024-08-28T04:11:37.692ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
19482024-08-28T04:11:37.731ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
19492024-08-28T04:11:37.788ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 97283 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:50960 req_id = 2450e67b-1a83-48a6-abc9-68229a2ac260 response_code = 204 uri = /switch-ports
19502024-08-28T04:11:37.789ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 167 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:50960 req_id = 7b73a1c1-ba86-422b-bee8-aceb74f7729f response_code = 200 uri = /network-bootstore-config
19512024-08-28T04:11:39.328ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:35837
19522024-08-28T04:11:39.328ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 73 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:35837 req_id = 60c35620-2c95-43c6-bab1-a3c1596f719f response_code = 200 uri = /baseboard
19532024-08-28T04:11:46.978ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 9011 local_addr = [fd00:1122:3344:101::1]:60627 method = GET remote_addr = [fd00:1122:3344:101::d]:48245 req_id = ab656c1e-7a8d-4174-ab8b-caf00e7bc605 response_code = 200 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f uri = /f62e5a13-51f3-4379-acf3-2f8e8601ac1f
19542024-08-28T04:12:00.709ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:39799
19552024-08-28T04:12:00.710ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 914 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:39799 req_id = dd7a8994-c87a-4c44-a4eb-2bb2e72716ff response_code = 200 uri = /v2p
19562024-08-28T04:12:00.710ZINFOSledAgent (PortManager): Mapping virtual NIC to physical host file = illumos-utils/src/opte/port_manager.rs:778 mapping = VirtualNetworkInterfaceHost { virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 250, 46, 104])), physical_host_ip: fd00:1122:3344:101::1, vni: Vni(9741316) }
19572024-08-28T04:12:00.710ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 89 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:39799 req_id = 9fcb92d2-9575-4faf-931c-4b5aa907bf7f response_code = 204 uri = /v2p
19582024-08-28T04:12:00.939ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:55347
19592024-08-28T04:12:00.953ZINFOSledAgent (InstanceManager): ensuring instance is registered file = sled-agent/src/instance_manager.rs:570 hardware = InstanceHardware { properties: InstanceProperties { ncpus: InstanceCpuCount(2), memory: ByteCount(1073741824), hostname: Hostname("localshark") }, nics: [NetworkInterface { id: 58fe1906-17a8-480a-b7ee-403178be6663, kind: Instance { id: 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 250, 46, 104])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(9741316), primary: true, slot: 0, transit_ips: [] }], source_nat: SourceNatConfig { ip: 10.151.2.175, first_port: 0, last_port: 16383 }, ephemeral_ip: Some(10.151.2.176), floating_ips: [], firewall_rules: [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 58fe1906-17a8-480a-b7ee-403178be6663, kind: Instance { id: 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 250, 46, 104])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(9741316), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 58fe1906-17a8-480a-b7ee-403178be6663, kind: Instance { id: 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 250, 46, 104])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(9741316), primary: true, slot: 0, transit_ips: [] }], filter_hosts: Some([Vpc(Vni(9741316))]), filter_ports: None, filter_protocols: None, action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 58fe1906-17a8-480a-b7ee-403178be6663, kind: Instance { id: 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 250, 46, 104])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(9741316), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(22), last: L4Port(22) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }], dhcp_config: DhcpConfig { dns_servers: [1.1.1.1, 9.9.9.9], host_domain: None, search_domains: [] }, disks: [DiskRequest { device: "nvme", name: "disk-4d41261df23", read_only: false, slot: Slot(0), volume_construction_request: Volume { block_size: 512, id: 4685fc71-6e85-4609-943c-747057e2242f, read_only_parent: Some(Volume { block_size: 512, id: b241ffb1-cc17-46f7-aa98-233d0dc6cca6, read_only_parent: None, sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 2, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: b391cd70-d3bf-4a92-9b5d-cf281b192d9b, key: Some("HrXNFJbWdWeNmB7r6EbS9Aq8I7VV3BX1ETsoAwnw9so="), key_pem: None, lossy: false, read_only: true, root_cert_pem: None, target: ["[fd00:1122:3344:101::16]:19001", "[fd00:1122:3344:101::17]:19001", "[fd00:1122:3344:101::12]:19001"] } }] }), sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 1, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 4685fc71-6e85-4609-943c-747057e2242f, key: Some("831JPVa5vx1Hcr2sdq3bVoiz6kb7ovp7exOtDV0hvoU="), key_pem: None, lossy: false, read_only: false, root_cert_pem: None, target: ["[fd00:1122:3344:101::17]:19002", "[fd00:1122:3344:101::19]:19001", "[fd00:1122:3344:101::14]:19000"] } }] } }], cloud_init_bytes: Some(..) } instance_id = 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b instance_runtime = InstanceRuntimeState { propolis_id: Some(208e8b27-c88a-499b-a882-9c02530056ef (propolis)), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2024-08-28T04:12:00.443549482Z } metadata = InstanceMetadata { silo_id: e3abc6f9-763c-486e-9205-7ca93e3777ea, project_id: 254921c9-b0a7-47c5-8d77-6316b5d12c19 } propolis_addr = [fd00:1122:3344:101::1:0]:12400 propolis_id = 208e8b27-c88a-499b-a882-9c02530056ef vmm_runtime = VmmRuntimeState { state: Starting, gen: Generation(1), time_updated: 2024-08-28T04:12:00.399638Z }
19602024-08-28T04:12:00.953ZINFOSledAgent (InstanceManager): registering new instance file = sled-agent/src/instance_manager.rs:603 instance_id = 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b propolis_id = 208e8b27-c88a-499b-a882-9c02530056ef
19612024-08-28T04:12:00.954ZINFOSledAgent (InstanceManager): initializing new Instance file = sled-agent/src/instance.rs:991 instance_id = 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b migration_id = None propolis_id = 208e8b27-c88a-499b-a882-9c02530056ef state = InstanceInitialState { hardware: InstanceHardware { properties: InstanceProperties { ncpus: InstanceCpuCount(2), memory: ByteCount(1073741824), hostname: Hostname("localshark") }, nics: [NetworkInterface { id: 58fe1906-17a8-480a-b7ee-403178be6663, kind: Instance { id: 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 250, 46, 104])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(9741316), primary: true, slot: 0, transit_ips: [] }], source_nat: SourceNatConfig { ip: 10.151.2.175, first_port: 0, last_port: 16383 }, ephemeral_ip: Some(10.151.2.176), floating_ips: [], firewall_rules: [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 58fe1906-17a8-480a-b7ee-403178be6663, kind: Instance { id: 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 250, 46, 104])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(9741316), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 58fe1906-17a8-480a-b7ee-403178be6663, kind: Instance { id: 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 250, 46, 104])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(9741316), primary: true, slot: 0, transit_ips: [] }], filter_hosts: Some([Vpc(Vni(9741316))]), filter_ports: None, filter_protocols: None, action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 58fe1906-17a8-480a-b7ee-403178be6663, kind: Instance { id: 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 250, 46, 104])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(9741316), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(22), last: L4Port(22) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }], dhcp_config: DhcpConfig { dns_servers: [1.1.1.1, 9.9.9.9], host_domain: None, search_domains: [] }, disks: [DiskRequest { device: "nvme", name: "disk-4d41261df23", read_only: false, slot: Slot(0), volume_construction_request: Volume { block_size: 512, id: 4685fc71-6e85-4609-943c-747057e2242f, read_only_parent: Some(Volume { block_size: 512, id: b241ffb1-cc17-46f7-aa98-233d0dc6cca6, read_only_parent: None, sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 2, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: b391cd70-d3bf-4a92-9b5d-cf281b192d9b, key: Some("HrXNFJbWdWeNmB7r6EbS9Aq8I7VV3BX1ETsoAwnw9so="), key_pem: None, lossy: false, read_only: true, root_cert_pem: None, target: ["[fd00:1122:3344:101::16]:19001", "[fd00:1122:3344:101::17]:19001", "[fd00:1122:3344:101::12]:19001"] } }] }), sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 1, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 4685fc71-6e85-4609-943c-747057e2242f, key: Some("831JPVa5vx1Hcr2sdq3bVoiz6kb7ovp7exOtDV0hvoU="), key_pem: None, lossy: false, read_only: false, root_cert_pem: None, target: ["[fd00:1122:3344:101::17]:19002", "[fd00:1122:3344:101::19]:19001", "[fd00:1122:3344:101::14]:19000"] } }] } }], cloud_init_bytes: Some(..) }, vmm_runtime: VmmRuntimeState { state: Starting, gen: Generation(1), time_updated: 2024-08-28T04:12:00.399638Z }, propolis_addr: [fd00:1122:3344:101::1:0]:12400, migration_id: None }
19622024-08-28T04:12:00.955ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 15607 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:55347 req_id = f01f1976-3feb-4319-8d18-99e50be41b18 response_code = 200 uri = /vmms/208e8b27-c88a-499b-a882-9c02530056ef
19632024-08-28T04:12:01.049ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:56222
19642024-08-28T04:12:01.051ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1588 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:56222 req_id = 76283ccf-772c-4f6f-8cc4-a14579718da4 response_code = 200 uri = /vmms/208e8b27-c88a-499b-a882-9c02530056ef/state
19652024-08-28T04:12:01.052ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte6", ip: 172.30.0.5, mac: MacAddr6([168, 64, 37, 250, 46, 104]), slot: 0, vni: Vni { inner: 9741316 }, subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.0.1 } }) }
19662024-08-28T04:12:01.052ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
19672024-08-28T04:12:01.096ZINFOSledAgent (InstanceManager): Configuring new Omicron zone: oxz_propolis-server_208e8b27-c88a-499b-a882-9c02530056ef file = illumos-utils/src/zone.rs:316 instance_id = 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b propolis_id = 208e8b27-c88a-499b-a882-9c02530056ef
19682024-08-28T04:12:01.119ZINFOSledAgent (InstanceManager): Installing Omicron zone: oxz_propolis-server_208e8b27-c88a-499b-a882-9c02530056ef file = illumos-utils/src/zone.rs:354 instance_id = 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b propolis_id = 208e8b27-c88a-499b-a882-9c02530056ef
19692024-08-28T04:12:02.883ZINFOSledAgent (InstanceManager): Profile for oxz_propolis-server_208e8b27-c88a-499b-a882-9c02530056ef: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="system/illumos/propolis-server"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlInstance0'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::1:0'/> <propval type="astring" name="listen_port" value='12400'/> <propval type="astring" name="metric_addr" value='dns'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34 instance_id = 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b propolis_id = 208e8b27-c88a-499b-a882-9c02530056ef
19702024-08-28T04:12:02.883ZINFOSledAgent (InstanceManager): Booting oxz_propolis-server_208e8b27-c88a-499b-a882-9c02530056ef zone file = illumos-utils/src/running_zone.rs:488 instance_id = 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b propolis_id = 208e8b27-c88a-499b-a882-9c02530056ef zone = oxz_propolis-server_208e8b27-c88a-499b-a882-9c02530056ef
19712024-08-28T04:12:03.073ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_208e8b27-c88a-499b-a882-9c02530056ef") failed: Property not found. retry in 57.728388ms file = illumos-utils/src/svc.rs:36 instance_id = 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b propolis_id = 208e8b27-c88a-499b-a882-9c02530056ef zone = oxz_propolis-server_208e8b27-c88a-499b-a882-9c02530056ef
19722024-08-28T04:12:03.153ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_208e8b27-c88a-499b-a882-9c02530056ef") failed: Property not found. retry in 52.188581ms file = illumos-utils/src/svc.rs:36 instance_id = 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b propolis_id = 208e8b27-c88a-499b-a882-9c02530056ef zone = oxz_propolis-server_208e8b27-c88a-499b-a882-9c02530056ef
19732024-08-28T04:12:03.221ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_208e8b27-c88a-499b-a882-9c02530056ef") failed: Property not found. retry in 254.835882ms file = illumos-utils/src/svc.rs:36 instance_id = 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b propolis_id = 208e8b27-c88a-499b-a882-9c02530056ef zone = oxz_propolis-server_208e8b27-c88a-499b-a882-9c02530056ef
19742024-08-28T04:12:03.494ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_208e8b27-c88a-499b-a882-9c02530056ef") failed: Property not found. retry in 418.888987ms file = illumos-utils/src/svc.rs:36 instance_id = 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b propolis_id = 208e8b27-c88a-499b-a882-9c02530056ef zone = oxz_propolis-server_208e8b27-c88a-499b-a882-9c02530056ef
19752024-08-28T04:12:03.925ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_208e8b27-c88a-499b-a882-9c02530056ef") failed: Property not found. retry in 611.813386ms file = illumos-utils/src/svc.rs:36 instance_id = 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b propolis_id = 208e8b27-c88a-499b-a882-9c02530056ef zone = oxz_propolis-server_208e8b27-c88a-499b-a882-9c02530056ef
19762024-08-28T04:12:04.551ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_208e8b27-c88a-499b-a882-9c02530056ef") failed: Property not found. retry in 920.020237ms file = illumos-utils/src/svc.rs:36 instance_id = 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b propolis_id = 208e8b27-c88a-499b-a882-9c02530056ef zone = oxz_propolis-server_208e8b27-c88a-499b-a882-9c02530056ef
19772024-08-28T04:12:05.484ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_208e8b27-c88a-499b-a882-9c02530056ef") failed: Property not found. retry in 1.341056194s file = illumos-utils/src/svc.rs:36 instance_id = 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b propolis_id = 208e8b27-c88a-499b-a882-9c02530056ef zone = oxz_propolis-server_208e8b27-c88a-499b-a882-9c02530056ef
19782024-08-28T04:12:06.541ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:59953
19792024-08-28T04:12:06.632ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:42545
19802024-08-28T04:12:06.632ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 48 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:42545 req_id = 64989041-077d-4aa7-bd2a-99e1ec3ccb09 response_code = 200 uri = /vpc-routes
19812024-08-28T04:12:06.793ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:52411
19822024-08-28T04:12:06.794ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 173 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:52411 req_id = 8bc1576f-1b76-4587-bf6d-0ec5cf905037 response_code = 200 uri = /v2p
19832024-08-28T04:12:06.823ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:45635
19842024-08-28T04:12:06.824ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
19852024-08-28T04:12:06.865ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
19862024-08-28T04:12:06.925ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 101863 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:45635 req_id = 54d211a1-05c8-4241-8dfa-20b1efc87d45 response_code = 204 uri = /switch-ports
19872024-08-28T04:12:06.930ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 716 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:45635 req_id = 68932825-d530-4001-b7f0-8a3275e91fd3 response_code = 200 uri = /network-bootstore-config
19882024-08-28T04:12:06.931ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2132 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:42545 req_id = 8490ee87-3085-43fc-a0c5-842e1d2ec448 response_code = 204 uri = /vpc-routes
19892024-08-28T04:12:06.935ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_208e8b27-c88a-499b-a882-9c02530056ef") failed: Property not found. retry in 1.393603592s file = illumos-utils/src/svc.rs:36 instance_id = 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b propolis_id = 208e8b27-c88a-499b-a882-9c02530056ef zone = oxz_propolis-server_208e8b27-c88a-499b-a882-9c02530056ef
19902024-08-28T04:12:07.321ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:51082
19912024-08-28T04:12:07.321ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 100 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:51082 req_id = 70451f1f-e1ec-463e-abb8-5c0887f1777f response_code = 200 uri = /vpc-routes
19922024-08-28T04:12:07.338ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:53865
19932024-08-28T04:12:07.338ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 47 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:53865 req_id = a422403a-820a-4745-9e23-6917403bed0b response_code = 200 uri = /vpc-routes
19942024-08-28T04:12:07.340ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:43029
19952024-08-28T04:12:07.358ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 131 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:51082 req_id = 165e7f76-9d65-4676-b173-96be6716f625 response_code = 204 uri = /vpc-routes
19962024-08-28T04:12:07.386ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:42131
19972024-08-28T04:12:07.402ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 120 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:53865 req_id = bfbaebff-1953-446c-93de-f540d9aa28cc response_code = 204 uri = /vpc-routes
19982024-08-28T04:12:07.433ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:39794
19992024-08-28T04:12:07.433ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 134 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:39794 req_id = 3471e8ae-3022-4216-8ef4-022902cb400f response_code = 200 uri = /v2p
20002024-08-28T04:12:07.462ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:42154
20012024-08-28T04:12:07.462ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
20022024-08-28T04:12:07.504ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
20032024-08-28T04:12:07.569ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 106396 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:42154 req_id = 9c13df6c-0889-4692-9142-fbf2aaa70df4 response_code = 204 uri = /switch-ports
20042024-08-28T04:12:07.569ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:49626
20052024-08-28T04:12:07.569ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 194 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:49626 req_id = 04546145-0dc8-4c62-967a-e180d6eb9f9f response_code = 200 uri = /v2p
20062024-08-28T04:12:07.569ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 114 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:42154 req_id = 884f6536-cfb6-44a3-b406-a7e7faf022ec response_code = 200 uri = /network-bootstore-config
20072024-08-28T04:12:07.640ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:47826
20082024-08-28T04:12:07.640ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
20092024-08-28T04:12:07.680ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
20102024-08-28T04:12:07.737ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 96955 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:47826 req_id = 87601d69-b868-4029-a03e-fa14a65d5010 response_code = 204 uri = /switch-ports
20112024-08-28T04:12:07.738ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 99 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:47826 req_id = 54a7076a-d546-4de8-806b-2cc3b2c7e12a response_code = 200 uri = /network-bootstore-config
20122024-08-28T04:12:08.341ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_208e8b27-c88a-499b-a882-9c02530056ef") failed: Property not found. retry in 884.90314ms file = illumos-utils/src/svc.rs:36 instance_id = 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b propolis_id = 208e8b27-c88a-499b-a882-9c02530056ef zone = oxz_propolis-server_208e8b27-c88a-499b-a882-9c02530056ef
20132024-08-28T04:12:09.258ZINFOSledAgent (InstanceManager): Started propolis in zone: oxz_propolis-server_208e8b27-c88a-499b-a882-9c02530056ef file = sled-agent/src/instance.rs:1449 instance_id = 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b propolis_id = 208e8b27-c88a-499b-a882-9c02530056ef
20142024-08-28T04:12:09.267ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_208e8b27-c88a-499b-a882-9c02530056ef") failed: Property not found. retry in 55.519509ms file = illumos-utils/src/svc.rs:36 instance_id = 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b propolis_id = 208e8b27-c88a-499b-a882-9c02530056ef
20152024-08-28T04:12:09.333ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_208e8b27-c88a-499b-a882-9c02530056ef") failed: Property not found. retry in 124.303789ms file = illumos-utils/src/svc.rs:36 instance_id = 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b propolis_id = 208e8b27-c88a-499b-a882-9c02530056ef
20162024-08-28T04:12:09.362ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:52343
20172024-08-28T04:12:09.362ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 81 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:52343 req_id = 91f27893-de64-4571-9bd9-d2bd0ff6682e response_code = 200 uri = /baseboard
20182024-08-28T04:12:09.470ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_208e8b27-c88a-499b-a882-9c02530056ef") failed: Property not found. retry in 216.899678ms file = illumos-utils/src/svc.rs:36 instance_id = 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b propolis_id = 208e8b27-c88a-499b-a882-9c02530056ef
20192024-08-28T04:12:09.698ZINFOSledAgent (InstanceManager): Propolis SMF service is online file = sled-agent/src/instance.rs:1458 instance_id = 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b propolis_id = 208e8b27-c88a-499b-a882-9c02530056ef
20202024-08-28T04:12:09.743ZINFOSledAgent (InstanceManager): Propolis HTTP server online file = sled-agent/src/instance.rs:1482 instance_id = 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b propolis_id = 208e8b27-c88a-499b-a882-9c02530056ef
20212024-08-28T04:12:09.883ZINFOSledAgent (InstanceManager): result of instance_ensure call is Ok(InstanceEnsureResponse { migrate: None }) file = sled-agent/src/instance.rs:727 instance_id = 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b propolis_id = 208e8b27-c88a-499b-a882-9c02530056ef
20222024-08-28T04:12:09.884ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3343248 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:59953 req_id = c4d1b850-f661-4bca-bb84-493e889335d0 response_code = 200 uri = /vmms/208e8b27-c88a-499b-a882-9c02530056ef/state
20232024-08-28T04:12:09.884ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2544549 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:43029 req_id = 4fb51d44-e5b9-4ca9-b906-610a682333d4 response_code = 200 uri = /vmms/208e8b27-c88a-499b-a882-9c02530056ef/state
20242024-08-28T04:12:09.884ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2497909 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:42131 req_id = 795ed5fc-1921-4b84-84eb-cd768639f5a6 response_code = 200 uri = /vmms/208e8b27-c88a-499b-a882-9c02530056ef/state
20252024-08-28T04:12:09.884ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Creating), migration_in: None, migration_out: None, time: 2024-08-28T04:12:09.884783222Z } file = sled-agent/src/instance.rs:594 instance_id = 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b propolis_id = 208e8b27-c88a-499b-a882-9c02530056ef
20262024-08-28T04:12:09.884ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:616 instance_id = 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b new_vmm_state = VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2024-08-28T04:12:09.884783222Z } propolis_id = 208e8b27-c88a-499b-a882-9c02530056ef
20272024-08-28T04:12:09.887ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:514 instance_id = 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b propolis_id = 208e8b27-c88a-499b-a882-9c02530056ef state = SledVmmState { vmm_state: VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2024-08-28T04:12:09.884783222Z }, migration_in: None, migration_out: None }
20282024-08-28T04:12:09.948ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Running), migration_in: None, migration_out: None, time: 2024-08-28T04:12:09.948587837Z } file = sled-agent/src/instance.rs:594 instance_id = 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b propolis_id = 208e8b27-c88a-499b-a882-9c02530056ef
20292024-08-28T04:12:09.948ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:616 instance_id = 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b new_vmm_state = VmmRuntimeState { state: Running, gen: Generation(3), time_updated: 2024-08-28T04:12:09.948587837Z } propolis_id = 208e8b27-c88a-499b-a882-9c02530056ef
20302024-08-28T04:12:09.948ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:514 instance_id = 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b propolis_id = 208e8b27-c88a-499b-a882-9c02530056ef state = SledVmmState { vmm_state: VmmRuntimeState { state: Running, gen: Generation(3), time_updated: 2024-08-28T04:12:09.948587837Z }, migration_in: None, migration_out: None }
20312024-08-28T04:12:16.977ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 6958 local_addr = [fd00:1122:3344:101::1]:60627 method = GET remote_addr = [fd00:1122:3344:101::d]:48245 req_id = 662a6524-a80e-40a6-8486-1a58312e1a66 response_code = 200 sled_id = f62e5a13-51f3-4379-acf3-2f8e8601ac1f uri = /f62e5a13-51f3-4379-acf3-2f8e8601ac1f
20322024-08-28T04:12:18.520ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:59903
20332024-08-28T04:12:18.521ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 607 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:59903 req_id = e00cb14a-f785-407b-a431-67acadfdbfcd response_code = 200 uri = /vmms/208e8b27-c88a-499b-a882-9c02530056ef/state
20342024-08-28T04:12:18.521ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Stopping), migration_in: None, migration_out: None, time: 2024-08-28T04:12:18.521778520Z } file = sled-agent/src/instance.rs:594 instance_id = 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b propolis_id = 208e8b27-c88a-499b-a882-9c02530056ef
20352024-08-28T04:12:18.521ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:616 instance_id = 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b new_vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(5), time_updated: 2024-08-28T04:12:18.521778520Z } propolis_id = 208e8b27-c88a-499b-a882-9c02530056ef
20362024-08-28T04:12:18.521ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:514 instance_id = 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b propolis_id = 208e8b27-c88a-499b-a882-9c02530056ef state = SledVmmState { vmm_state: VmmRuntimeState { state: Stopping, gen: Generation(5), time_updated: 2024-08-28T04:12:18.521778520Z }, migration_in: None, migration_out: None }
20372024-08-28T04:12:18.587ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Destroyed), migration_in: None, migration_out: None, time: 2024-08-28T04:12:18.587083968Z } file = sled-agent/src/instance.rs:594 instance_id = 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b propolis_id = 208e8b27-c88a-499b-a882-9c02530056ef
20382024-08-28T04:12:18.587ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:616 instance_id = 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b new_vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(6), time_updated: 2024-08-28T04:12:18.587083968Z } propolis_id = 208e8b27-c88a-499b-a882-9c02530056ef
20392024-08-28T04:12:18.587ZINFOSledAgent (InstanceManager): terminating VMM that has exited file = sled-agent/src/instance.rs:630 instance_id = 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b propolis_id = 208e8b27-c88a-499b-a882-9c02530056ef
20402024-08-28T04:12:18.587ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
20412024-08-28T04:12:18.587ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
20422024-08-28T04:12:18.596ZINFOSledAgent (ZoneBundler): creating zone bundle context = ZoneBundleContext { storage_dirs: ["/pool/int/4dad4c76-eb6c-4281-9f1d-a63fdb0117e1/debug/bundle/zone", "/pool/int/51625293-e5a8-4cc4-9640-b64176e7a48e/debug/bundle/zone"], cause: TerminatedInstance, extra_log_dirs: ["/pool/ext/57750eb9-c738-4eb8-89f2-c5eab164d34c/crypt/debug", "/pool/ext/0f906b25-a03c-4334-807e-07a343a51361/crypt/debug", "/pool/ext/d6170dc9-a1d4-44a9-ba24-00c07a403b1e/crypt/debug", "/pool/ext/8955d8a9-1b85-47ea-bd5c-275ef6dc0b1a/crypt/debug", "/pool/ext/fb7c10cb-4fd0-49dd-8c21-f83feafbd7dc/crypt/debug", "/pool/ext/ba3c519e-f98a-4ee2-a345-a21845f5f7a4/crypt/debug", "/pool/ext/0913dfa8-99f2-4518-8d37-827ea32c30ba/crypt/debug", "/pool/ext/ed7027bd-3cf9-4764-bc15-404c842ab02a/crypt/debug", "/pool/ext/e3fa46c7-4f3a-45f7-afd8-b7799f191587/crypt/debug"] } file = sled-agent/src/zone_bundle.rs:339 zone_name = oxz_propolis-server_208e8b27-c88a-499b-a882-9c02530056ef
20432024-08-28T04:12:19.323ZINFOSledAgent (ZoneBundler): finished zone bundle file = sled-agent/src/zone_bundle.rs:1137 metadata = ZoneBundleMetadata { id: ZoneBundleId { zone_name: "oxz_propolis-server_208e8b27-c88a-499b-a882-9c02530056ef", bundle_id: dc32f087-00e8-41fc-a4c9-7ed7cb6316ba }, time_created: 2024-08-28T04:12:18.596879866Z, version: 0, cause: TerminatedInstance }
20442024-08-28T04:12:19.323ZWARNSledAgent (InstanceManager): Halting and removing zone: oxz_propolis-server_208e8b27-c88a-499b-a882-9c02530056ef file = sled-agent/src/instance.rs:826 instance_id = 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b propolis_id = 208e8b27-c88a-499b-a882-9c02530056ef
20452024-08-28T04:12:21.093ZINFOSledAgent (InstanceManager): halt_and_remove_logged: Previous zone state: Running file = illumos-utils/src/zone.rs:270 instance_id = 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b propolis_id = 208e8b27-c88a-499b-a882-9c02530056ef
20462024-08-28T04:12:21.117ZINFOSledAgent (PortManager): Removed route set for subnet file = illumos-utils/src/opte/port_manager.rs:904 id = RouterId { vni: Vni(9741316), kind: System }
20472024-08-28T04:12:21.117ZINFOSledAgent (PortManager): Removed route set for subnet file = illumos-utils/src/opte/port_manager.rs:904 id = RouterId { vni: Vni(9741316), kind: Custom(V4(Ipv4Net { addr: 172.30.0.0, width: 22 })) }
20482024-08-28T04:12:21.118ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:514 instance_id = 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b propolis_id = 208e8b27-c88a-499b-a882-9c02530056ef state = SledVmmState { vmm_state: VmmRuntimeState { state: Destroyed, gen: Generation(7), time_updated: 2024-08-28T04:12:21.118108352Z }, migration_in: None, migration_out: None }
20492024-08-28T04:12:21.154ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:514 instance_id = 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b propolis_id = 208e8b27-c88a-499b-a882-9c02530056ef state = SledVmmState { vmm_state: VmmRuntimeState { state: Destroyed, gen: Generation(7), time_updated: 2024-08-28T04:12:21.118108352Z }, migration_in: None, migration_out: None }
20502024-08-28T04:12:21.159ZINFOSledAgent (InstanceManager): State monitoring task complete file = sled-agent/src/instance.rs:763 instance_id = 533126e7-adf4-4e3a-b0c9-69b6af4c2b1b propolis_id = 208e8b27-c88a-499b-a882-9c02530056ef
20512024-08-28T04:12:21.778ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:39359
20522024-08-28T04:12:21.778ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 48 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:39359 req_id = eb6d0e86-95ab-4df9-80c9-5d836b1a6714 response_code = 200 uri = /vpc-routes
20532024-08-28T04:12:21.786ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 98 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:39359 req_id = f26f7810-2997-4026-a9a0-071af78be226 response_code = 204 uri = /vpc-routes
20542024-08-28T04:12:21.818ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:39089
20552024-08-28T04:12:21.818ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 102 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:39089 req_id = 1d2aa781-450b-488f-90f7-3908d17e4955 response_code = 200 uri = /v2p
20562024-08-28T04:12:22.301ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:58140
20572024-08-28T04:12:22.302ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 161 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:58140 req_id = c0f89b84-180e-4899-abdb-9378b6fb25dc response_code = 200 uri = /v2p
20582024-08-28T04:12:24.198ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:61823
20592024-08-28T04:12:24.198ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 153 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:61823 req_id = 965d20a3-f8c3-4780-951f-5d019898bcee response_code = 200 uri = /vpc-routes
20602024-08-28T04:12:24.206ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 125 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:61823 req_id = 329d63d8-6e8a-43ea-9df2-606a038cfff5 response_code = 204 uri = /vpc-routes