Max level shown:
1 [ Sep 1 20:56:02 Enabled. ]
2 [ Sep 1 20:56:02 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/sled-agent/sled-agent run /opt/oxide/sled-agent/pkg/config.toml &"). ]
3 [ Sep 1 20:56:02 Method "start" exited with status 0. ]
4 [ Sep 1 20:56:02 Rereading configuration. ]
5 [ Sep 1 20:56:03 No 'refresh' method defined. Treating as :true. ]
6 note: configured to log to "/dev/stdout"
72024-09-01T20:56:03.007ZINFOSledAgent: Ensuring contract reaper thread file = illumos-utils/src/running_zone.rs:113
82024-09-01T20:56:03.008ZINFOSledAgent (BootstrapAgentStartup): finding links ["net0", "net1"] file = sled-agent/src/bootstrap/pre_server.rs:255
92024-09-01T20:56:03.061ZINFOSledAgent (BootstrapAgentStartup): Starting mg-ddm service addr-objs = [AddrObject { interface: "net0", name: "ll" }, AddrObject { interface: "net1", name: "ll" }] file = sled-agent/src/bootstrap/pre_server.rs:262
102024-09-01T20:56:03.061ZINFOSledAgent (BootstrapAgentStartup): Importing mg-ddm service file = sled-agent/src/bootstrap/maghemite.rs:42 path = /opt/oxide/mg-ddm/pkg/ddm/manifest.xml
112024-09-01T20:56:03.175ZINFOSledAgent (BootstrapAgentStartup): Setting mg-ddm interfaces file = sled-agent/src/bootstrap/maghemite.rs:50 interfaces = ("net0/ll" "net1/ll")
122024-09-01T20:56:03.184ZINFOSledAgent (BootstrapAgentStartup): Enabling mg-ddm service file = sled-agent/src/bootstrap/maghemite.rs:56
132024-09-01T20:56:03.192ZINFOSledAgent (BootstrapAgentStartup): Ensuring zfs key directory exists file = sled-agent/src/bootstrap/pre_server.rs:272 path = /var/run/oxide/
142024-09-01T20:56:03.795ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fdb0:18c0:4d0d:a02a::, width: 64 }
152024-09-01T20:56:03.934ZINFOSledAgent: Starting KeyManager file = sled-agent/src/long_running_tasks.rs:126
162024-09-01T20:56:03.934ZINFOSledAgent: Starting StorageManager file = sled-agent/src/long_running_tasks.rs:138
172024-09-01T20:56:03.934ZINFOSledAgent: Starting StorageMonitor file = sled-agent/src/long_running_tasks.rs:151
182024-09-01T20:56:03.934ZINFOSledAgent: Starting HardwareManager file = sled-agent/src/long_running_tasks.rs:173 nongimlet_observed_disks = [] sled_mode = Scrimlet { asic: SoftNpuZone }
192024-09-01T20:56:03.934ZINFOSledAgent (DumpSetup-worker): DumpSetup poll loop started. file = sled-agent/src/dump_setup.rs:520
202024-09-01T20:56:03.934ZINFOSledAgent (HardwareManager): Creating HardwareManager file = sled-hardware/src/illumos/mod.rs:716
212024-09-01T20:56:03.937ZINFOSledAgent (HardwareManager): Generated i86pc baseboard Pc { identifier: "sock", model: "i86pc" } file = sled-hardware/src/illumos/mod.rs:623
222024-09-01T20:56:03.937ZWARNSledAgent (HardwareManager): Device is not a Gimlet (i86pc), proceeding with null hardware view file = sled-hardware/src/illumos/mod.rs:765
232024-09-01T20:56:03.937ZINFOSledAgent: Starting HardwareMonitor file = sled-agent/src/long_running_tasks.rs:187
242024-09-01T20:56:03.937ZINFOSledAgent (HardwareManager): Monitoring for hardware updates file = sled-hardware/src/illumos/mod.rs:824
252024-09-01T20:56:03.937ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/m2_0.vdev
262024-09-01T20:56:03.937ZINFOSledAgent (HardwareMonitor): Checking current full hardware snapshot disks = {} file = sled-agent/src/hardware_monitor.rs:271 underlay_network_info = None
272024-09-01T20:56:03.937ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:451
282024-09-01T20:56:03.937ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
292024-09-01T20:56:03.937ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } path = /scratch/m2_0.vdev
302024-09-01T20:56:03.948ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/m2_0.vdev file = sled-hardware/src/disk.rs:360
312024-09-01T20:56:03.948ZINFOSledAgent (StorageResources): Formatting zpool with generated ID file = sled-hardware/src/disk.rs:371 id = fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e (zpool)
322024-09-01T20:56:04.048ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:174 zpool = ZpoolName { id: fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e (zpool), kind: Internal }
332024-09-01T20:56:04.447ZINFOSledAgent (StorageManager): Received DetectedRawDisksChanged { raw_disks: {}, tx: ... } file = sled-storage/src/manager.rs:451
342024-09-01T20:56:04.447ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/m2_1.vdev
352024-09-01T20:56:04.447ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
362024-09-01T20:56:04.447ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
372024-09-01T20:56:04.447ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:451
382024-09-01T20:56:04.447ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
392024-09-01T20:56:04.447ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" } path = /scratch/m2_1.vdev
402024-09-01T20:56:04.455ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/m2_1.vdev file = sled-hardware/src/disk.rs:360
412024-09-01T20:56:04.455ZINFOSledAgent (StorageResources): Formatting zpool with generated ID file = sled-hardware/src/disk.rs:371 id = c13a4209-582e-438f-b076-515144773dc5 (zpool)
422024-09-01T20:56:04.543ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:174 zpool = ZpoolName { id: c13a4209-582e-438f-b076-515144773dc5 (zpool), kind: Internal }
432024-09-01T20:56:04.947ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_0.vdev
442024-09-01T20:56:04.947ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: c13a4209-582e-438f-b076-515144773dc5 (zpool), kind: Internal } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
452024-09-01T20:56:04.947ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:451
462024-09-01T20:56:04.947ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
472024-09-01T20:56:04.947ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
482024-09-01T20:56:04.947ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_1.vdev
492024-09-01T20:56:04.947ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
502024-09-01T20:56:04.947ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:451
512024-09-01T20:56:04.947ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
522024-09-01T20:56:04.947ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: c13a4209-582e-438f-b076-515144773dc5 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
532024-09-01T20:56:04.947ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_2.vdev
542024-09-01T20:56:04.947ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
552024-09-01T20:56:04.947ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
562024-09-01T20:56:04.947ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:451
572024-09-01T20:56:04.947ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
582024-09-01T20:56:04.947ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: c13a4209-582e-438f-b076-515144773dc5 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
592024-09-01T20:56:04.947ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
602024-09-01T20:56:04.947ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
612024-09-01T20:56:04.947ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_3.vdev
622024-09-01T20:56:04.947ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:451
632024-09-01T20:56:04.947ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
642024-09-01T20:56:04.947ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: c13a4209-582e-438f-b076-515144773dc5 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
652024-09-01T20:56:04.947ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
662024-09-01T20:56:04.947ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
672024-09-01T20:56:04.947ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_4.vdev
682024-09-01T20:56:04.947ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: c13a4209-582e-438f-b076-515144773dc5 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
692024-09-01T20:56:04.947ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:451
702024-09-01T20:56:04.947ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
712024-09-01T20:56:04.947ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
722024-09-01T20:56:04.947ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
732024-09-01T20:56:04.947ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_5.vdev
742024-09-01T20:56:04.947ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: c13a4209-582e-438f-b076-515144773dc5 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
752024-09-01T20:56:04.947ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
762024-09-01T20:56:04.947ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
772024-09-01T20:56:04.947ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:451
782024-09-01T20:56:04.947ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
792024-09-01T20:56:04.947ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_6.vdev
802024-09-01T20:56:04.947ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: c13a4209-582e-438f-b076-515144773dc5 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
812024-09-01T20:56:04.947ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
822024-09-01T20:56:04.947ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
832024-09-01T20:56:04.947ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:451
842024-09-01T20:56:04.947ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
852024-09-01T20:56:04.947ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_7.vdev
862024-09-01T20:56:04.947ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:451
872024-09-01T20:56:04.947ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
882024-09-01T20:56:04.947ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: c13a4209-582e-438f-b076-515144773dc5 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
892024-09-01T20:56:04.947ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
902024-09-01T20:56:04.947ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_8.vdev
912024-09-01T20:56:04.947ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
922024-09-01T20:56:04.947ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:451
932024-09-01T20:56:04.947ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
942024-09-01T20:56:04.947ZINFOSledAgent: Waiting for boot disk file = sled-agent/src/long_running_tasks.rs:97
952024-09-01T20:56:04.947ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: c13a4209-582e-438f-b076-515144773dc5 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
962024-09-01T20:56:04.947ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
972024-09-01T20:56:04.947ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
982024-09-01T20:56:04.947ZINFOSledAgent: Found boot disk DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } file = sled-agent/src/long_running_tasks.rs:99
992024-09-01T20:56:04.947ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
1002024-09-01T20:56:04.947ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: c13a4209-582e-438f-b076-515144773dc5 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
1012024-09-01T20:56:04.947ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
1022024-09-01T20:56:04.947ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
1032024-09-01T20:56:04.947ZINFOSledAgent: Starting Bootstore file = sled-agent/src/long_running_tasks.rs:211
1042024-09-01T20:56:04.947ZINFOSledAgent (bootstore): No ledger in /pool/int/fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e/cluster/bootstore-fsm-state.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1052024-09-01T20:56:04.947ZINFOSledAgent (bootstore): No ledger in /pool/int/c13a4209-582e-438f-b076-515144773dc5/cluster/bootstore-fsm-state.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1062024-09-01T20:56:04.947ZINFOSledAgent (bootstore): No ledger found. Loading Fsm::State as Uninitialized file = bootstore/src/schemes/v0/storage.rs:83 peer_id = pc-sock-i86pc
1072024-09-01T20:56:04.947ZINFOSledAgent (bootstore): No ledger in /pool/int/fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e/cluster/bootstore-network-config.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1082024-09-01T20:56:04.947ZINFOSledAgent (bootstore): No ledger in /pool/int/c13a4209-582e-438f-b076-515144773dc5/cluster/bootstore-network-config.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1092024-09-01T20:56:04.947ZINFOSledAgent (bootstore): No ledger found for network config file = bootstore/src/schemes/v0/storage.rs:146 peer_id = pc-sock-i86pc
1102024-09-01T20:56:04.947ZINFOSledAgent: Starting Bootstore DDMD poller file = sled-agent/src/long_running_tasks.rs:216
1112024-09-01T20:56:04.947ZINFOSledAgent: Starting ZoneBundler related tasks file = sled-agent/src/long_running_tasks.rs:231
1122024-09-01T20:56:04.957ZINFOSledAgent (ServiceManager): Creating ServiceManager file = sled-agent/src/services.rs:730
1132024-09-01T20:56:04.957ZINFOSledAgent (auto-cleanup-task): top of bundle cleanup loop file = sled-agent/src/zone_bundle.rs:186 next_cleanup = Instant { tv_sec: 3582, tv_nsec: 392685594 } time_to_next_cleanup = 599.999924499s
1142024-09-01T20:56:04.957ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
1152024-09-01T20:56:04.957ZINFOSledAgent (ServiceManager): Ensuring scrimlet services (enabling services) file = sled-agent/src/services.rs:3866
1162024-09-01T20:56:04.957ZINFOSledAgent (BootstrapAgentStartup): No ledger in /pool/int/fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e/config/sled-agent-request.json file = common/src/ledger.rs:188
1172024-09-01T20:56:04.957ZINFOSledAgent (BootstrapAgentStartup): No ledger in /pool/int/c13a4209-582e-438f-b076-515144773dc5/config/sled-agent-request.json file = common/src/ledger.rs:188
1182024-09-01T20:56:04.960ZINFOSledAgent (dropshot (BootstrapAgent)): listening file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:205 local_addr = [fdb0:18c0:4d0d:a02a::1]:80
1192024-09-01T20:56:04.960ZINFOSledAgent (SledAgentSprocketsServer): Started listening file = sled-agent/src/bootstrap/sprockets_server.rs:44 local_addr = [fdb0:18c0:4d0d:a02a::1]:12346
1202024-09-01T20:56:04.961ZINFOSledAgent (RSS): Injecting RSS configuration: RackInitializeRequest { trust_quorum_peers: None, bootstrap_discovery: OnlyOurs, ntp_servers: [ "0.pool.ntp.org", ], dns_servers: [ 1.1.1.1, 9.9.9.9, ], internal_services_ip_pool_ranges: [ V4( Ipv4Range { first: 10.151.2.164, last: 10.151.2.173, }, ), ], external_dns_ips: [ 10.151.2.164, 10.151.2.165, ], external_dns_zone_name: "oxide.test", external_certificates: "<redacted>", recovery_silo: RecoverySiloConfig { silo_name: Name( "recovery", ), user_name: UserId( "recovery", ), user_password_hash: NewPasswordHash( PasswordHashString { string: "$argon2id$v=19$m=98304,t=13,p=1$RUlWc0ZxaHo0WFdrN0N6ZQ$S8p52j85GPvMhR/ek3GL0el/oProgTwWpHJZ8lsQQoY", encoding: B64, }, ), }, rack_network_config: RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [ PortConfigV2 { routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), nexthop: 10.151.2.100, vlan_id: None, local_pref: None, }, ], addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.2.174, width: 24, }, ), vlan_id: None, }, ], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: None, bgp_peers: [], autoneg: false, lldp: None, }, ], bgp: [], bfd: [], }, allowed_source_ips: Any, } file = sled-agent/src/rack_setup/service.rs:1053
1212024-09-01T20:56:04.961ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = internal-dns/src/resolver.rs:73
1222024-09-01T20:56:04.961ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
1232024-09-01T20:56:04.961ZINFOSledAgent (RSS): No ledger in /pool/int/fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e/config/rss-plan-completed.marker file = common/src/ledger.rs:188
1242024-09-01T20:56:04.961ZINFOSledAgent (RSS): No ledger in /pool/int/c13a4209-582e-438f-b076-515144773dc5/config/rss-plan-completed.marker file = common/src/ledger.rs:188
1252024-09-01T20:56:04.961ZINFOSledAgent (RSS): RSS configuration has not been fully applied yet file = sled-agent/src/rack_setup/service.rs:1113
1262024-09-01T20:56:04.961ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
1272024-09-01T20:56:04.961ZINFOSledAgent (RSS): No ledger in /pool/int/fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e/config/rss-sled-plan.json file = common/src/ledger.rs:188
1282024-09-01T20:56:04.961ZINFOSledAgent (RSS): No ledger in /pool/int/c13a4209-582e-438f-b076-515144773dc5/config/rss-sled-plan.json file = common/src/ledger.rs:188
1292024-09-01T20:56:04.961ZINFOSledAgent (RSS): Creating new allocation plan file = sled-agent/src/rack_setup/service.rs:1154
1302024-09-01T20:56:04.961ZINFOSledAgent (RSS): Serializing plan file = sled-agent/src/rack_setup/plan/sled.rs:155
1312024-09-01T20:56:04.961ZINFOSledAgent (RSS): Creating plan for the sled at fdb0:18c0:4d0d:a02a::1 file = sled-agent/src/rack_setup/plan/sled.rs:128
1322024-09-01T20:56:04.961ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
1332024-09-01T20:56:04.961ZINFOSledAgent (RSS): Writing ledger to /pool/int/fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e/config/.rss-sled-plan.json.tmp file = common/src/ledger.rs:199
1342024-09-01T20:56:04.961ZINFOSledAgent (RSS): Writing ledger to /pool/int/c13a4209-582e-438f-b076-515144773dc5/config/.rss-sled-plan.json.tmp file = common/src/ledger.rs:199
1352024-09-01T20:56:04.962ZINFOSledAgent (RSS): Sled plan written to storage: Plan { rack_id: 82c87802-7334-4dc0-9ec8-482884e1fd48, sleds: { [fdb0:18c0:4d0d:a02a::1]:12346: StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: d40a3584-3bdb-438a-9e24-58fb6b212748, rack_id: 82c87802-7334-4dc0-9ec8-482884e1fd48, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64, }, }, }, }, }, config: RackInitializeRequest { trust_quorum_peers: None, bootstrap_discovery: OnlyOurs, ntp_servers: [ "0.pool.ntp.org", ], dns_servers: [ 1.1.1.1, 9.9.9.9, ], internal_services_ip_pool_ranges: [ V4( Ipv4Range { first: 10.151.2.164, last: 10.151.2.173, }, ), ], external_dns_ips: [ 10.151.2.164, 10.151.2.165, ], external_dns_zone_name: "oxide.test", external_certificates: "<redacted>", recovery_silo: RecoverySiloConfig { silo_name: Name( "recovery", ), user_name: UserId( "recovery", ), user_password_hash: NewPasswordHash( PasswordHashString { string: "$argon2id$v=19$m=98304,t=13,p=1$RUlWc0ZxaHo0WFdrN0N6ZQ$S8p52j85GPvMhR/ek3GL0el/oProgTwWpHJZ8lsQQoY", encoding: B64, }, ), }, rack_network_config: RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [ PortConfigV2 { routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), nexthop: 10.151.2.100, vlan_id: None, local_pref: None, }, ], addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.2.174, width: 24, }, ), vlan_id: None, }, ], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: None, bgp_peers: [], autoneg: false, lldp: None, }, ], bgp: [], bfd: [], }, allowed_source_ips: Any, }, } file = sled-agent/src/rack_setup/plan/sled.rs:175
1362024-09-01T20:56:04.962ZINFOSledAgent (RSS): Writing Rack Network Configuration to bootstore file = sled-agent/src/rack_setup/service.rs:1187
1372024-09-01T20:56:04.962ZINFOSledAgent (bootstore): Attempting to update network config with generation: 1, current_generation: 0 file = bootstore/src/schemes/v0/peer.rs:564 peer_id = pc-sock-i86pc
1382024-09-01T20:56:04.962ZINFOSledAgent (bootstore): Writing ledger to /pool/int/fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
1392024-09-01T20:56:04.962ZINFOSledAgent (bootstore): Writing ledger to /pool/int/c13a4209-582e-438f-b076-515144773dc5/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
1402024-09-01T20:56:04.962ZINFOSledAgent (bootstore): Broadcasting network config with generation 1 file = bootstore/src/schemes/v0/peer.rs:648 peer_id = pc-sock-i86pc
1412024-09-01T20:56:04.962ZINFOSledAgent (BootstrapAgentRssHandler): Received initialization request from RSS file = sled-agent/src/bootstrap/rss_handle.rs:217 request = StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: d40a3584-3bdb-438a-9e24-58fb6b212748, rack_id: 82c87802-7334-4dc0-9ec8-482884e1fd48, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64 } } } } target_sled = [fdb0:18c0:4d0d:a02a::1]:12346
1422024-09-01T20:56:04.962ZINFOSledAgent (SledAgentSprocketsServer): Accepted connection file = sled-agent/src/bootstrap/sprockets_server.rs:66 remote_addr = [fdb0:18c0:4d0d:a02a::1]:42666
1432024-09-01T20:56:04.962ZINFOSledAgent (SledAgentMain): Loading Sled Agent: StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: d40a3584-3bdb-438a-9e24-58fb6b212748, rack_id: 82c87802-7334-4dc0-9ec8-482884e1fd48, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64 } } } } file = sled-agent/src/bootstrap/server.rs:359
1442024-09-01T20:56:04.962ZINFOSledAgent (SledAgentMain): KeyManager: using hardcoded secret retriever file = sled-agent/src/bootstrap/server.rs:374
1452024-09-01T20:56:04.962ZINFOSledAgent: setting up sled agent server file = sled-agent/src/server.rs:44
1462024-09-01T20:56:04.962ZINFOSledAgent (StorageManager): Received KeyManagerReady file = sled-storage/src/manager.rs:451
1472024-09-01T20:56:04.962ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = internal-dns/src/resolver.rs:73
1482024-09-01T20:56:04.962ZINFOSledAgent (StorageManager): Loading storage config file = sled-storage/src/manager.rs:644
1492024-09-01T20:56:04.962ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:101::, width: 64 }
1502024-09-01T20:56:04.962ZINFOSledAgent (StorageManager): No ledger in /pool/int/fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = load_disks_ledger
1512024-09-01T20:56:04.962ZINFOSledAgent (StorageManager): No ledger in /pool/int/c13a4209-582e-438f-b076-515144773dc5/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = load_disks_ledger
1522024-09-01T20:56:04.962ZINFOSledAgent (StorageManager): No ledger of physical disks exists file = sled-storage/src/manager.rs:633
1532024-09-01T20:56:04.962ZINFOSledAgent (StorageManager): KeyManager ready, but no ledger detected file = sled-storage/src/manager.rs:661
1542024-09-01T20:56:04.962ZINFOSledAgent (StorageManager): automatically managing disks file = sled-storage/src/manager.rs:440
1552024-09-01T20:56:04.962ZINFOSledAgent (StorageResources): Synchronizing disk managment file = sled-storage/src/resources.rs:302
1562024-09-01T20:56:04.962ZINFOSledAgent (StorageManager): Successfully synchronized disks without error file = sled-storage/src/manager.rs:532 result = DisksManagementResult { status: [] }
1572024-09-01T20:56:04.965ZINFOSledAgent (ServiceManager): Enabling switch zone (new) file = sled-agent/src/services.rs:4163
1582024-09-01T20:56:04.965ZINFOSledAgent (ServiceManager): Starting switch zone file = sled-agent/src/services.rs:4545
1592024-09-01T20:56:04.996ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
1602024-09-01T20:56:05.018ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_switch file = illumos-utils/src/zone.rs:316
1612024-09-01T20:56:05.035ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_switch file = illumos-utils/src/zone.rs:354
1622024-09-01T20:56:05.042ZINFOSledAgent: SledAgent::new(..) starting file = sled-agent/src/sled_agent.rs:388 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
1632024-09-01T20:56:05.042ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
1642024-09-01T20:56:05.042ZINFOSledAgent: Requested swap device of size 64 GiB file = sled-agent/src/sled_agent.rs:400 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
1652024-09-01T20:56:05.042ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1662024-09-01T20:56:05.045ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 74.466161ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "fb0ea327-a3a5-4314-9e70-d80ed41409bf", "content-length": "124", "date": "Sun, 01 Sep 2024 20:56:04 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "fb0ea327-a3a5-4314-9e70-d80ed41409bf" } file = clients/ddm-admin-client/src/lib.rs:177
1672024-09-01T20:56:05.054ZINFOSledAgent: attempting to create encrypted zvol: name="oxi_fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e/swap", size_gb=64 file = sled-agent/src/swap_device.rs:153
1682024-09-01T20:56:05.078ZINFOSledAgent: successfully created encrypted zvol: name="oxi_fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e/swap", size_gb=64 file = sled-agent/src/swap_device.rs:249
1692024-09-01T20:56:05.078ZINFOSledAgent: adding swap device: swapname="/dev/zvol/dsk/oxi_fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e/swap" file = sled-agent/src/swap_device.rs:96
1702024-09-01T20:56:05.083ZINFOSledAgent: Mounting backing filesystems file = sled-agent/src/sled_agent.rs:415 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
1712024-09-01T20:56:05.083ZINFOSledAgent (BackingFs): Processing fmd file = sled-agent/src/backing_fs.rs:127
1722024-09-01T20:56:05.083ZINFOSledAgent (BackingFs): Ensuring dataset oxi_fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e/backing/fmd file = sled-agent/src/backing_fs.rs:137
1732024-09-01T20:56:05.122ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1742024-09-01T20:56:05.122ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 100.073989ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "fb74d657-98ea-4e9c-bf4b-3767049d77f2", "content-length": "124", "date": "Sun, 01 Sep 2024 20:56:04 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "fb74d657-98ea-4e9c-bf4b-3767049d77f2" } file = clients/ddm-admin-client/src/lib.rs:177
1752024-09-01T20:56:05.172ZINFOSledAgent (BackingFs): Stopping service svc:/system/fmd:default file = sled-agent/src/backing_fs.rs:172
1762024-09-01T20:56:05.223ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1772024-09-01T20:56:05.224ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 145.148917ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "460c7e21-e2c5-4cf8-9a6a-6e1e1971b130", "content-length": "124", "date": "Sun, 01 Sep 2024 20:56:04 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "460c7e21-e2c5-4cf8-9a6a-6e1e1971b130" } file = clients/ddm-admin-client/src/lib.rs:177
1782024-09-01T20:56:05.370ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1792024-09-01T20:56:05.371ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 102.398909ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "2b6138b5-8efc-476c-8a46-2651d47b21ff", "content-length": "124", "date": "Sun, 01 Sep 2024 20:56:04 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "2b6138b5-8efc-476c-8a46-2651d47b21ff" } file = clients/ddm-admin-client/src/lib.rs:177
1802024-09-01T20:56:05.391ZINFOSledAgent (BackingFs): Mounting oxi_fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e/backing/fmd on /var/fm/fmd file = sled-agent/src/backing_fs.rs:180
1812024-09-01T20:56:05.404ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/rsrc file = sled-agent/src/backing_fs.rs:188
1822024-09-01T20:56:05.404ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/ckpt file = sled-agent/src/backing_fs.rs:188
1832024-09-01T20:56:05.404ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/xprt file = sled-agent/src/backing_fs.rs:188
1842024-09-01T20:56:05.404ZINFOSledAgent (BackingFs): Starting service svc:/system/fmd:default file = sled-agent/src/backing_fs.rs:194
1852024-09-01T20:56:05.474ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1862024-09-01T20:56:05.475ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 189.658069ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "5e97a51e-7df3-4450-bfb3-c91b54c3286d", "content-length": "124", "date": "Sun, 01 Sep 2024 20:56:04 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "5e97a51e-7df3-4450-bfb3-c91b54c3286d" } file = clients/ddm-admin-client/src/lib.rs:177
1872024-09-01T20:56:05.665ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1882024-09-01T20:56:05.666ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 252.828735ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "11492330-de5e-435c-b41a-712f64ef9751", "content-length": "124", "date": "Sun, 01 Sep 2024 20:56:04 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "11492330-de5e-435c-b41a-712f64ef9751" } file = clients/ddm-admin-client/src/lib.rs:177
1892024-09-01T20:56:05.919ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1902024-09-01T20:56:05.919ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 314.319223ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "7a83f4ba-4085-4381-a122-586d5e7a0a22", "content-length": "124", "date": "Sun, 01 Sep 2024 20:56:04 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "7a83f4ba-4085-4381-a122-586d5e7a0a22" } file = clients/ddm-admin-client/src/lib.rs:177
1912024-09-01T20:56:06.128ZINFOSledAgent: using '[AddrObject { interface: "net0", name: "ll" }, AddrObject { interface: "net1", name: "ll" }]' as data links for xde driver file = illumos-utils/src/opte/illumos.rs:90 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
1922024-09-01T20:56:06.133ZINFOSledAgent (dropshot): listening file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:205 local_addr = [fd00:1122:3344:101::1]:46126 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
1932024-09-01T20:56:06.133ZINFOSledAgent (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = internal-dns/src/resolver.rs:73 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
1942024-09-01T20:56:06.133ZINFOSledAgent (producer-server): starting oximeter metric producer server address = [fd00:1122:3344:101::1]:46126 file = oximeter/producer/src/lib.rs:283 interval = 30s producer_id = d40a3584-3bdb-438a-9e24-58fb6b212748 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
1952024-09-01T20:56:06.135ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 206.426919ms error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
1962024-09-01T20:56:06.135ZINFOSledAgent (VmmReservoirManager): 50% of 102952026112 physical ram = 51476013056 bytes) file = sled-agent/src/vmm_reservoir.rs:267 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
1972024-09-01T20:56:06.135ZINFOSledAgent (VmmReservoirManager): Setting reservoir size to 49090 MiB bytes file = sled-agent/src/vmm_reservoir.rs:275 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
1982024-09-01T20:56:06.135ZINFOSledAgent (ServiceManager): sled agent started file = sled-agent/src/services.rs:914 underlay_address = fd00:1122:3344:101::1
1992024-09-01T20:56:06.135ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = internal-dns/src/resolver.rs:73
2002024-09-01T20:56:06.235ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
2012024-09-01T20:56:06.343ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 635.392234ms error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
2022024-09-01T20:56:06.980ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 1.343395461s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
2032024-09-01T20:56:08.325ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 1.877770333s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
2042024-09-01T20:56:08.670ZINFOSledAgent (ServiceManager): Setting up dendrite service file = sled-agent/src/services.rs:2639
2052024-09-01T20:56:08.670ZINFOSledAgent (ServiceManager): Setting up lldpd service file = sled-agent/src/services.rs:2857
2062024-09-01T20:56:08.670ZINFOSledAgent (ServiceManager): Setting up MGS service file = sled-agent/src/services.rs:2509
2072024-09-01T20:56:08.670ZINFOSledAgent (ServiceManager): Setting up wicketd service file = sled-agent/src/services.rs:2556
2082024-09-01T20:56:08.670ZINFOSledAgent (ServiceManager): Setting up mgd service file = sled-agent/src/services.rs:2938
2092024-09-01T20:56:08.670ZINFOSledAgent (ServiceManager): Setting up mg-ddm service file = sled-agent/src/services.rs:2980
2102024-09-01T20:56:08.672ZINFOSledAgent (ServiceManager): Setting up tfport service file = sled-agent/src/services.rs:2782
2112024-09-01T20:56:08.673ZINFOSledAgent (ServiceManager): Setting up Simulated SP service file = sled-agent/src/services.rs:2550
2122024-09-01T20:56:08.674ZINFOSledAgent (ServiceManager): Profile for oxz_switch: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService0'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/mgs"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="address" value='[::1]:12225'/> <propval type="astring" name="id" value='fd35ac89-65cf-4bb0-8a9a-1c2572e374de'/> <propval type="astring" name="rack_id" value='82c87802-7334-4dc0-9ec8-482884e1fd48'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/wicketd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="address" value='[::1]:12226'/> <propval type="astring" name="artifact-address" value='[fdb0:18c0:4d0d:a02a::2]:12227'/> <propval type="astring" name="baseboard-file" value='/opt/oxide/baseboard.json'/> <propval type="astring" name="mgs-address" value='[::1]:12225'/> <propval type="astring" name="nexus-proxy-address" value='[::]:12229'/> <propval type="astring" name="rack-subnet" value='fd00:1122:3344::'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/switch_zone_setup"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="gz_local_link_addr" value='fe80::8:20ff:feea:3866'/> <property type="astring" name="link_local_links"> <astring_list> <value_node value="tfportrear0_0" /> <value_node value="oxBootstrap0" /> </astring_list> </property> <propval type="astring" name="bootstrap_addr" value='fdb0:18c0:4d0d:a02a::2'/> <propval type="astring" name="bootstrap_vnic" value='oxBootstrap0'/> <propval type="astring" name="baseboard_info" value='{"type":"pc","identifier":"sock","model":"i86pc"}'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/dendrite"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="sled_id" value='d40a3584-3bdb-438a-9e24-58fb6b212748'/> <propval type="astring" name="rack_id" value='82c87802-7334-4dc0-9ec8-482884e1fd48'/> <propval type="astring" name="address" value='[::1]:12224'/> <propval type="astring" name="front_ports" value='1'/> <propval type="astring" name="rear_ports" value='1'/> <propval type="astring" name="port_config" value='/opt/oxide/dendrite/misc/softnpu_single_sled_config.toml'/> <propval type="astring" name="mgmt" value='uds'/> <propval type="astring" name="uds_path" value='/opt/softnpu/stuff'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/tfport"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="host" value='[::1]'/> <propval type="astring" name="port" value='12224'/> <propval type="astring" name="flags" value='--sync-only'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/lldpd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="board_rev" value='softnpu_front_1_rear_1'/> <propval type="astring" name="scrimlet_id" value='sock'/> <propval type="astring" name="scrimlet_model" value='i86pc'/> <propval type="astring" name="address" value='[::1]:12230'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/pumpkind"> </service> <service version="1" type="service" name="oxide/mgd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="sled_uuid" value='d40a3584-3bdb-438a-9e24-58fb6b212748'/> <propval type="astring" name="rack_uuid" value='82c87802-7334-4dc0-9ec8-482884e1fd48'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/mg-ddm"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="mode" value='transit'/> <propval type="astring" name="dendrite" value='true'/> <propval type="astring" name="sled_uuid" value='d40a3584-3bdb-438a-9e24-58fb6b212748'/> <propval type="astring" name="rack_uuid" value='82c87802-7334-4dc0-9ec8-482884e1fd48'/> <propval type="astring" name="interfaces" value='tfportrear0_0/ll'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/uplink"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
2132024-09-01T20:56:08.674ZINFOSledAgent (ServiceManager): Booting oxz_switch zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_switch
2142024-09-01T20:56:08.957ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 62.240238ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2152024-09-01T20:56:09.154ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 98.815016ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2162024-09-01T20:56:09.341ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 144.836799ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2172024-09-01T20:56:09.513ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 251.196519ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2182024-09-01T20:56:09.883ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 793.744293ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2192024-09-01T20:56:10.204ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 2.511284729s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
2202024-09-01T20:56:10.837ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 1.021874676s file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2212024-09-01T20:56:12.059ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 943.200923ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2222024-09-01T20:56:12.717ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 7.930487741s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
2232024-09-01T20:56:13.206ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 1.064672994s file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2242024-09-01T20:56:14.727ZINFOSledAgent (dropshot (SledAgent)): listening file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:205 local_addr = [fd00:1122:3344:101::1]:12345
2252024-09-01T20:56:14.727ZINFOSledAgent (SledAgentMain): Sled Agent loaded; recording configuration file = sled-agent/src/bootstrap/server.rs:429
2262024-09-01T20:56:14.727ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
2272024-09-01T20:56:14.727ZINFOSledAgent (SledAgentMain): Writing ledger to /pool/int/fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e/config/.sled-agent-request.json.tmp file = common/src/ledger.rs:199
2282024-09-01T20:56:14.727ZINFOSledAgent (SledAgentMain): Writing ledger to /pool/int/c13a4209-582e-438f-b076-515144773dc5/config/.sled-agent-request.json.tmp file = common/src/ledger.rs:199
2292024-09-01T20:56:14.728ZINFOSledAgent (HardwareMonitor): Sled Agent Started file = sled-agent/src/hardware_monitor.rs:138
2302024-09-01T20:56:14.728ZINFOSledAgent (HardwareMonitor): Checking current full hardware snapshot disks = {} file = sled-agent/src/hardware_monitor.rs:271 underlay_network_info = Some((fd00:1122:3344:101::2, Some(RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56 }, infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [PortConfigV2 { routes: [RouteConfig { destination: V4(Ipv4Net { addr: 0.0.0.0, width: 0 }), nexthop: 10.151.2.100, vlan_id: None, local_pref: None }], addresses: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: None, bgp_peers: [], autoneg: false, lldp: None }], bgp: [], bfd: [] })))
2312024-09-01T20:56:14.728ZINFOSledAgent (SledAgentSprocketsServer): Connection closed file = sled-agent/src/bootstrap/sprockets_server.rs:73 remote_addr = [fdb0:18c0:4d0d:a02a::1]:42666
2322024-09-01T20:56:14.728ZINFOSledAgent (ServiceManager): Ensuring scrimlet services (enabling services) file = sled-agent/src/services.rs:3866
2332024-09-01T20:56:14.969ZINFOSledAgent (BootstrapAgentRssHandler): Peer agent initialized file = sled-agent/src/bootstrap/rss_handle.rs:108 peer = [fdb0:18c0:4d0d:a02a::1]:12346
2342024-09-01T20:56:14.969ZINFOSledAgent (BootstrapAgentRssHandler): Initialized sled agent file = sled-agent/src/bootstrap/rss_handle.rs:232 target_sled = [fdb0:18c0:4d0d:a02a::1]:12346
2352024-09-01T20:56:14.969ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
2362024-09-01T20:56:14.970ZINFOSledAgent (RSS): No ledger in /pool/int/fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e/config/rss-service-plan-v4.json file = common/src/ledger.rs:188
2372024-09-01T20:56:14.970ZINFOSledAgent (RSS): No ledger in /pool/int/c13a4209-582e-438f-b076-515144773dc5/config/rss-service-plan-v4.json file = common/src/ledger.rs:188
2382024-09-01T20:56:14.970ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
2392024-09-01T20:56:14.970ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
2402024-09-01T20:56:14.970ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
2412024-09-01T20:56:14.974ZINFOSledAgent (ServiceManager): Re-enabling running switch zone (new address) file = sled-agent/src/services.rs:4187 new = [fd00:1122:3344:101::2, ::1] old = [::1]
2422024-09-01T20:56:14.974ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::2 exists file = sled-agent/src/services.rs:4203
2432024-09-01T20:56:14.974ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:101::2, prefix: 64 })) file = illumos-utils/src/running_zone.rs:534 zone = oxz_switch
2442024-09-01T20:56:15.260ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:34278
2452024-09-01T20:56:15.261ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
2462024-09-01T20:56:15.261ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 191 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:34278 req_id = 776e654a-e55b-4fe8-9698-228cdb41d2ca response_code = 200 uri = /inventory
2472024-09-01T20:56:15.292ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:36278
2482024-09-01T20:56:15.292ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 75 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:36278 req_id = 24e200bb-965b-4106-bc15-9db5cbc6f594 response_code = 200 uri = /sled-role
2492024-09-01T20:56:15.293ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
2502024-09-01T20:56:15.293ZINFOSledAgent (RSS): Writing ledger to /pool/int/fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e/config/.rss-service-plan-v4.json.tmp file = common/src/ledger.rs:199
2512024-09-01T20:56:15.294ZINFOSledAgent (RSS): Writing ledger to /pool/int/c13a4209-582e-438f-b076-515144773dc5/config/.rss-service-plan-v4.json.tmp file = common/src/ledger.rs:199
2522024-09-01T20:56:15.294ZINFOSledAgent (RSS): Service plan written to storage file = sled-agent/src/rack_setup/plan/service.rs:1001
2532024-09-01T20:56:15.326ZINFOSledAgent (RSS): attempting to set up sled's storage: OmicronPhysicalDisksConfig { generation: Generation(1), disks: [OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, id: 78e2510c-e5ba-4e25-be31-d2f0d48a2ad9, pool_id: 067f6720-0053-47f5-be0b-12fc815674a2 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, id: cac7a945-399d-41d6-bb34-28decee9026e, pool_id: c65e07fa-2bd7-44d6-b309-46d959ab18fb (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, id: 773e7b21-9b8c-4338-a590-2a16a8e9daa4, pool_id: e8cbb923-594d-495a-af32-03297bdb9d06 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, id: 8172d9b8-3ca4-418d-980c-23a0a15def63, pool_id: b7b09d01-4b5a-46f4-a346-2e7b6a0e3462 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, id: ee525c03-590f-4a6a-9b70-fb7394981a51, pool_id: 14681c40-9950-4e30-8f61-ec7a2ca60e1c (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, id: e45c6785-0737-4481-bded-678a4e168fca, pool_id: 8d3c930e-5dba-4325-82af-75dd024bc8cc (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, id: cb59dc24-5df6-4bf5-beb0-fd92d67b66f7, pool_id: 6421be57-8f17-44b8-8266-2192ac87bae0 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, id: 750a63c1-5c98-4e12-98d5-030b231efa02, pool_id: 64079151-57af-4617-9647-1f77fe70ee41 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, id: 98f4807c-0da6-4e52-9ab6-9c14e6f591d7, pool_id: 7a19116b-f2a7-4643-8d6b-df9db59b345a (zpool) }] } file = sled-agent/src/rack_setup/service.rs:355 sled_address = [fd00:1122:3344:101::1]:12345
2542024-09-01T20:56:15.326ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:35310
2552024-09-01T20:56:15.326ZINFOSledAgent: physical disks ensure file = sled-agent/src/sled_agent.rs:856 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
2562024-09-01T20:56:15.326ZINFOSledAgent (StorageManager): Received OmicronPhysicalDisksEnsure { config: OmicronPhysicalDisksConfig { generation: Generation(1), disks: [OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, id: 78e2510c-e5ba-4e25-be31-d2f0d48a2ad9, pool_id: 067f6720-0053-47f5-be0b-12fc815674a2 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, id: cac7a945-399d-41d6-bb34-28decee9026e, pool_id: c65e07fa-2bd7-44d6-b309-46d959ab18fb (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, id: 773e7b21-9b8c-4338-a590-2a16a8e9daa4, pool_id: e8cbb923-594d-495a-af32-03297bdb9d06 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, id: 8172d9b8-3ca4-418d-980c-23a0a15def63, pool_id: b7b09d01-4b5a-46f4-a346-2e7b6a0e3462 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, id: ee525c03-590f-4a6a-9b70-fb7394981a51, pool_id: 14681c40-9950-4e30-8f61-ec7a2ca60e1c (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, id: e45c6785-0737-4481-bded-678a4e168fca, pool_id: 8d3c930e-5dba-4325-82af-75dd024bc8cc (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, id: cb59dc24-5df6-4bf5-beb0-fd92d67b66f7, pool_id: 6421be57-8f17-44b8-8266-2192ac87bae0 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, id: 750a63c1-5c98-4e12-98d5-030b231efa02, pool_id: 64079151-57af-4617-9647-1f77fe70ee41 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, id: 98f4807c-0da6-4e52-9ab6-9c14e6f591d7, pool_id: 7a19116b-f2a7-4643-8d6b-df9db59b345a (zpool) }] }, tx: ... } file = sled-storage/src/manager.rs:451
2572024-09-01T20:56:15.326ZINFOSledAgent (StorageManager): No ledger in /pool/int/fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = omicron_physical_disks_ensure
2582024-09-01T20:56:15.326ZINFOSledAgent (StorageManager): No ledger in /pool/int/c13a4209-582e-438f-b076-515144773dc5/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = omicron_physical_disks_ensure
2592024-09-01T20:56:15.326ZINFOSledAgent (StorageManager): No previously-stored 'requested disks', creating new ledger file = sled-storage/src/manager.rs:866 request = omicron_physical_disks_ensure
2602024-09-01T20:56:15.326ZINFOSledAgent (StorageResources): Synchronizing disk managment file = sled-storage/src/resources.rs:302
2612024-09-01T20:56:15.326ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:342
2622024-09-01T20:56:15.326ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
2632024-09-01T20:56:15.326ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } path = /scratch/u2_0.vdev
2642024-09-01T20:56:15.480ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_0.vdev file = sled-hardware/src/disk.rs:360
2652024-09-01T20:56:15.480ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = 067f6720-0053-47f5-be0b-12fc815674a2 (zpool)
2662024-09-01T20:56:15.730ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/d40a3584-3bdb-438a-9e24-58fb6b212748): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/probe_manager.rs:230 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
2672024-09-01T20:56:16.949ZINFOSledAgent (VmmReservoirManager): Finished setting reservoir size to 49090 MiB bytes file = sled-agent/src/vmm_reservoir.rs:279 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
2682024-09-01T20:56:16.950ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/d40a3584-3bdb-438a-9e24-58fb6b212748): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/probe_manager.rs:230 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
2692024-09-01T20:56:16.959ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:174 zpool = ZpoolName { id: 067f6720-0053-47f5-be0b-12fc815674a2 (zpool), kind: External }
2702024-09-01T20:56:16.967ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:218
2712024-09-01T20:56:16.967ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:220
2722024-09-01T20:56:16.967ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:224
2732024-09-01T20:56:16.967ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:226
2742024-09-01T20:56:16.967ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key")
2752024-09-01T20:56:16.967ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key")
2762024-09-01T20:56:16.967ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:238
2772024-09-01T20:56:17.003ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
2782024-09-01T20:56:17.011ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_067f6720-0053-47f5-be0b-12fc815674a2/crypt/zone file = sled-storage/src/dataset.rs:281
2792024-09-01T20:56:17.181ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_067f6720-0053-47f5-be0b-12fc815674a2 file = sled-storage/src/dataset.rs:367
2802024-09-01T20:56:17.188ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:440
2812024-09-01T20:56:17.188ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:357
2822024-09-01T20:56:17.188ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:342
2832024-09-01T20:56:17.188ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
2842024-09-01T20:56:17.188ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } path = /scratch/u2_1.vdev
2852024-09-01T20:56:17.198ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_1.vdev file = sled-hardware/src/disk.rs:360
2862024-09-01T20:56:17.198ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = c65e07fa-2bd7-44d6-b309-46d959ab18fb (zpool)
2872024-09-01T20:56:17.292ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:174 zpool = ZpoolName { id: c65e07fa-2bd7-44d6-b309-46d959ab18fb (zpool), kind: External }
2882024-09-01T20:56:17.299ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:218
2892024-09-01T20:56:17.299ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:220
2902024-09-01T20:56:17.299ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:224
2912024-09-01T20:56:17.299ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:226
2922024-09-01T20:56:17.299ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key")
2932024-09-01T20:56:17.300ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key")
2942024-09-01T20:56:17.300ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:238
2952024-09-01T20:56:17.334ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
2962024-09-01T20:56:17.342ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_c65e07fa-2bd7-44d6-b309-46d959ab18fb/crypt/zone file = sled-storage/src/dataset.rs:281
2972024-09-01T20:56:17.511ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_c65e07fa-2bd7-44d6-b309-46d959ab18fb file = sled-storage/src/dataset.rs:367
2982024-09-01T20:56:17.519ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:440
2992024-09-01T20:56:17.519ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:357
3002024-09-01T20:56:17.519ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:342
3012024-09-01T20:56:17.519ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3022024-09-01T20:56:17.519ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } path = /scratch/u2_2.vdev
3032024-09-01T20:56:17.529ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_2.vdev file = sled-hardware/src/disk.rs:360
3042024-09-01T20:56:17.529ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = e8cbb923-594d-495a-af32-03297bdb9d06 (zpool)
3052024-09-01T20:56:17.623ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/dataset.rs:174 zpool = ZpoolName { id: e8cbb923-594d-495a-af32-03297bdb9d06 (zpool), kind: External }
3062024-09-01T20:56:17.631ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/dataset.rs:218
3072024-09-01T20:56:17.631ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:220
3082024-09-01T20:56:17.631ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:224
3092024-09-01T20:56:17.631ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:226
3102024-09-01T20:56:17.631ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key")
3112024-09-01T20:56:17.631ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key")
3122024-09-01T20:56:17.631ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:238
3132024-09-01T20:56:17.667ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3142024-09-01T20:56:17.675ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_e8cbb923-594d-495a-af32-03297bdb9d06/crypt/zone file = sled-storage/src/dataset.rs:281
3152024-09-01T20:56:17.852ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_e8cbb923-594d-495a-af32-03297bdb9d06 file = sled-storage/src/dataset.rs:367
3162024-09-01T20:56:17.859ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:440
3172024-09-01T20:56:17.859ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:357
3182024-09-01T20:56:17.859ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:342
3192024-09-01T20:56:17.859ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3202024-09-01T20:56:17.859ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } path = /scratch/u2_3.vdev
3212024-09-01T20:56:17.869ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_3.vdev file = sled-hardware/src/disk.rs:360
3222024-09-01T20:56:17.869ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = b7b09d01-4b5a-46f4-a346-2e7b6a0e3462 (zpool)
3232024-09-01T20:56:17.966ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/dataset.rs:174 zpool = ZpoolName { id: b7b09d01-4b5a-46f4-a346-2e7b6a0e3462 (zpool), kind: External }
3242024-09-01T20:56:17.974ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/dataset.rs:218
3252024-09-01T20:56:17.974ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:220
3262024-09-01T20:56:17.974ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:224
3272024-09-01T20:56:17.974ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:226
3282024-09-01T20:56:17.974ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key")
3292024-09-01T20:56:17.974ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key")
3302024-09-01T20:56:17.974ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:238
3312024-09-01T20:56:18.009ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3322024-09-01T20:56:18.013ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/d40a3584-3bdb-438a-9e24-58fb6b212748): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/probe_manager.rs:230 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
3332024-09-01T20:56:18.017ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_b7b09d01-4b5a-46f4-a346-2e7b6a0e3462/crypt/zone file = sled-storage/src/dataset.rs:281
3342024-09-01T20:56:18.193ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_b7b09d01-4b5a-46f4-a346-2e7b6a0e3462 file = sled-storage/src/dataset.rs:367
3352024-09-01T20:56:18.200ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:440
3362024-09-01T20:56:18.200ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:357
3372024-09-01T20:56:18.200ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:342
3382024-09-01T20:56:18.200ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3392024-09-01T20:56:18.200ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } path = /scratch/u2_4.vdev
3402024-09-01T20:56:18.210ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_4.vdev file = sled-hardware/src/disk.rs:360
3412024-09-01T20:56:18.210ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = 14681c40-9950-4e30-8f61-ec7a2ca60e1c (zpool)
3422024-09-01T20:56:18.305ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/dataset.rs:174 zpool = ZpoolName { id: 14681c40-9950-4e30-8f61-ec7a2ca60e1c (zpool), kind: External }
3432024-09-01T20:56:18.312ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/dataset.rs:218
3442024-09-01T20:56:18.312ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:220
3452024-09-01T20:56:18.312ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:224
3462024-09-01T20:56:18.312ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:226
3472024-09-01T20:56:18.313ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key")
3482024-09-01T20:56:18.313ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key")
3492024-09-01T20:56:18.313ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:238
3502024-09-01T20:56:18.348ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3512024-09-01T20:56:18.356ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_14681c40-9950-4e30-8f61-ec7a2ca60e1c/crypt/zone file = sled-storage/src/dataset.rs:281
3522024-09-01T20:56:18.533ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_14681c40-9950-4e30-8f61-ec7a2ca60e1c file = sled-storage/src/dataset.rs:367
3532024-09-01T20:56:18.541ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:440
3542024-09-01T20:56:18.541ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:357
3552024-09-01T20:56:18.541ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:342
3562024-09-01T20:56:18.541ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3572024-09-01T20:56:18.541ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } path = /scratch/u2_5.vdev
3582024-09-01T20:56:18.551ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_5.vdev file = sled-hardware/src/disk.rs:360
3592024-09-01T20:56:18.551ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = 8d3c930e-5dba-4325-82af-75dd024bc8cc (zpool)
3602024-09-01T20:56:18.643ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/dataset.rs:174 zpool = ZpoolName { id: 8d3c930e-5dba-4325-82af-75dd024bc8cc (zpool), kind: External }
3612024-09-01T20:56:18.650ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/dataset.rs:218
3622024-09-01T20:56:18.650ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:220
3632024-09-01T20:56:18.650ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:224
3642024-09-01T20:56:18.650ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:226
3652024-09-01T20:56:18.650ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key")
3662024-09-01T20:56:18.651ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key")
3672024-09-01T20:56:18.651ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:238
3682024-09-01T20:56:18.683ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3692024-09-01T20:56:18.691ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_8d3c930e-5dba-4325-82af-75dd024bc8cc/crypt/zone file = sled-storage/src/dataset.rs:281
3702024-09-01T20:56:18.856ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_8d3c930e-5dba-4325-82af-75dd024bc8cc file = sled-storage/src/dataset.rs:367
3712024-09-01T20:56:18.864ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:440
3722024-09-01T20:56:18.864ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:357
3732024-09-01T20:56:18.865ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:342
3742024-09-01T20:56:18.865ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3752024-09-01T20:56:18.865ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } path = /scratch/u2_6.vdev
3762024-09-01T20:56:18.874ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_6.vdev file = sled-hardware/src/disk.rs:360
3772024-09-01T20:56:18.874ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = 6421be57-8f17-44b8-8266-2192ac87bae0 (zpool)
3782024-09-01T20:56:18.963ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/dataset.rs:174 zpool = ZpoolName { id: 6421be57-8f17-44b8-8266-2192ac87bae0 (zpool), kind: External }
3792024-09-01T20:56:18.970ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/dataset.rs:218
3802024-09-01T20:56:18.970ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:220
3812024-09-01T20:56:18.970ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:224
3822024-09-01T20:56:18.970ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:226
3832024-09-01T20:56:18.970ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key")
3842024-09-01T20:56:18.971ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key")
3852024-09-01T20:56:18.971ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:238
3862024-09-01T20:56:19.009ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3872024-09-01T20:56:19.017ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/d40a3584-3bdb-438a-9e24-58fb6b212748): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/probe_manager.rs:230 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
3882024-09-01T20:56:19.017ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_6421be57-8f17-44b8-8266-2192ac87bae0/crypt/zone file = sled-storage/src/dataset.rs:281
3892024-09-01T20:56:19.175ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_6421be57-8f17-44b8-8266-2192ac87bae0 file = sled-storage/src/dataset.rs:367
3902024-09-01T20:56:19.182ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:440
3912024-09-01T20:56:19.182ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:357
3922024-09-01T20:56:19.182ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:342
3932024-09-01T20:56:19.182ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3942024-09-01T20:56:19.182ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } path = /scratch/u2_7.vdev
3952024-09-01T20:56:19.192ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_7.vdev file = sled-hardware/src/disk.rs:360
3962024-09-01T20:56:19.192ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = 64079151-57af-4617-9647-1f77fe70ee41 (zpool)
3972024-09-01T20:56:19.285ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/dataset.rs:174 zpool = ZpoolName { id: 64079151-57af-4617-9647-1f77fe70ee41 (zpool), kind: External }
3982024-09-01T20:56:19.293ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/dataset.rs:218
3992024-09-01T20:56:19.293ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:220
4002024-09-01T20:56:19.293ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:224
4012024-09-01T20:56:19.293ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:226
4022024-09-01T20:56:19.293ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key")
4032024-09-01T20:56:19.293ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key")
4042024-09-01T20:56:19.293ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:238
4052024-09-01T20:56:19.329ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
4062024-09-01T20:56:19.338ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_64079151-57af-4617-9647-1f77fe70ee41/crypt/zone file = sled-storage/src/dataset.rs:281
4072024-09-01T20:56:19.517ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_64079151-57af-4617-9647-1f77fe70ee41 file = sled-storage/src/dataset.rs:367
4082024-09-01T20:56:19.525ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:440
4092024-09-01T20:56:19.525ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:357
4102024-09-01T20:56:19.525ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:342
4112024-09-01T20:56:19.525ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
4122024-09-01T20:56:19.525ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } path = /scratch/u2_8.vdev
4132024-09-01T20:56:19.534ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_8.vdev file = sled-hardware/src/disk.rs:360
4142024-09-01T20:56:19.534ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = 7a19116b-f2a7-4643-8d6b-df9db59b345a (zpool)
4152024-09-01T20:56:19.639ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/dataset.rs:174 zpool = ZpoolName { id: 7a19116b-f2a7-4643-8d6b-df9db59b345a (zpool), kind: External }
4162024-09-01T20:56:19.646ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/dataset.rs:218
4172024-09-01T20:56:19.646ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:220
4182024-09-01T20:56:19.646ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:224
4192024-09-01T20:56:19.646ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:226
4202024-09-01T20:56:19.646ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key")
4212024-09-01T20:56:19.647ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key")
4222024-09-01T20:56:19.647ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:238
4232024-09-01T20:56:19.683ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
4242024-09-01T20:56:19.690ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_7a19116b-f2a7-4643-8d6b-df9db59b345a/crypt/zone file = sled-storage/src/dataset.rs:281
4252024-09-01T20:56:19.866ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_7a19116b-f2a7-4643-8d6b-df9db59b345a file = sled-storage/src/dataset.rs:367
4262024-09-01T20:56:19.874ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:440
4272024-09-01T20:56:19.874ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:357
4282024-09-01T20:56:19.874ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e/config/.omicron-physical-disks.json.tmp file = common/src/ledger.rs:199 request = omicron_physical_disks_ensure
4292024-09-01T20:56:19.874ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: c13a4209-582e-438f-b076-515144773dc5 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 067f6720-0053-47f5-be0b-12fc815674a2 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: c65e07fa-2bd7-44d6-b309-46d959ab18fb (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: e8cbb923-594d-495a-af32-03297bdb9d06 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: b7b09d01-4b5a-46f4-a346-2e7b6a0e3462 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 14681c40-9950-4e30-8f61-ec7a2ca60e1c (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 8d3c930e-5dba-4325-82af-75dd024bc8cc (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 6421be57-8f17-44b8-8266-2192ac87bae0 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 64079151-57af-4617-9647-1f77fe70ee41 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 7a19116b-f2a7-4643-8d6b-df9db59b345a (zpool), kind: External } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
4302024-09-01T20:56:19.874ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
4312024-09-01T20:56:19.874ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
4322024-09-01T20:56:19.874ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/c13a4209-582e-438f-b076-515144773dc5/config/.omicron-physical-disks.json.tmp file = common/src/ledger.rs:199 request = omicron_physical_disks_ensure
4332024-09-01T20:56:19.875ZINFOSledAgent: physical disks ensure: Updated storage file = sled-agent/src/sled_agent.rs:860 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
4342024-09-01T20:56:19.875ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
4352024-09-01T20:56:19.875ZINFOSledAgent: physical disks ensure: Propagating new generation of disks file = sled-agent/src/sled_agent.rs:886 generation = Generation(1) sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
4362024-09-01T20:56:19.875ZINFOSledAgent: physical disks ensure: Updated storage monitor file = sled-agent/src/sled_agent.rs:891 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
4372024-09-01T20:56:19.875ZINFOSledAgent: physical disks ensure: Updated zone bundler file = sled-agent/src/sled_agent.rs:896 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
4382024-09-01T20:56:19.875ZINFOSledAgent (ProbeManager): use_only_these_disks: Processing new request file = sled-agent/src/probe_manager.rs:121 gen = Generation(1) sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
4392024-09-01T20:56:19.875ZINFOSledAgent: physical disks ensure: Updated probes file = sled-agent/src/sled_agent.rs:902 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
4402024-09-01T20:56:19.875ZINFOSledAgent (InstanceManager): use_only_these_disks: Processing new request file = sled-agent/src/instance_manager.rs:803 gen = Generation(1)
4412024-09-01T20:56:19.875ZINFOSledAgent: physical disks ensure: Updated instances file = sled-agent/src/sled_agent.rs:907 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
4422024-09-01T20:56:19.875ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 4548926 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:35310 req_id = 0dbfa954-152b-453e-bbfc-911a7959ce31 response_code = 200 uri = /omicron-physical-disks
4432024-09-01T20:56:19.906ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(2), zones: [OmicronZoneConfig { id: 4410131d-d543-4f5e-b042-2b8f7df286aa, underlay_address: fd00:1122:3344:1::1, filesystem_pool: Some(ZpoolName { id: 067f6720-0053-47f5-be0b-12fc815674a2 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 067f6720-0053-47f5-be0b-12fc815674a2 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: c56694eb-8314-46fd-8d68-a8cdc82c2423, underlay_address: fd00:1122:3344:2::1, filesystem_pool: Some(ZpoolName { id: c65e07fa-2bd7-44d6-b309-46d959ab18fb (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: c65e07fa-2bd7-44d6-b309-46d959ab18fb (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 871d4f62-0910-41a9-b1f8-74b5d8646a65, underlay_address: fd00:1122:3344:3::1, filesystem_pool: Some(ZpoolName { id: e8cbb923-594d-495a-af32-03297bdb9d06 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: e8cbb923-594d-495a-af32-03297bdb9d06 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }] } file = sled-agent/src/rack_setup/service.rs:438 sled_address = [fd00:1122:3344:101::1]:12345
4442024-09-01T20:56:19.906ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:51413
4452024-09-01T20:56:19.907ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 4410131d-d543-4f5e-b042-2b8f7df286aa, dataset_name: DatasetName { pool_name: ZpoolName { id: 067f6720-0053-47f5-be0b-12fc815674a2 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
4462024-09-01T20:56:19.907ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 4410131d-d543-4f5e-b042-2b8f7df286aa, dataset_name: DatasetName { pool_name: ZpoolName { id: 067f6720-0053-47f5-be0b-12fc815674a2 (zpool), kind: External }, kind: InternalDns }, responder: ... }
4472024-09-01T20:56:19.955ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c56694eb-8314-46fd-8d68-a8cdc82c2423, dataset_name: DatasetName { pool_name: ZpoolName { id: c65e07fa-2bd7-44d6-b309-46d959ab18fb (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
4482024-09-01T20:56:19.955ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: c56694eb-8314-46fd-8d68-a8cdc82c2423, dataset_name: DatasetName { pool_name: ZpoolName { id: c65e07fa-2bd7-44d6-b309-46d959ab18fb (zpool), kind: External }, kind: InternalDns }, responder: ... }
4492024-09-01T20:56:20.004ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 871d4f62-0910-41a9-b1f8-74b5d8646a65, dataset_name: DatasetName { pool_name: ZpoolName { id: e8cbb923-594d-495a-af32-03297bdb9d06 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
4502024-09-01T20:56:20.004ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 871d4f62-0910-41a9-b1f8-74b5d8646a65, dataset_name: DatasetName { pool_name: ZpoolName { id: e8cbb923-594d-495a-af32-03297bdb9d06 (zpool), kind: External }, kind: InternalDns }, responder: ... }
4512024-09-01T20:56:20.019ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/d40a3584-3bdb-438a-9e24-58fb6b212748): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/probe_manager.rs:230 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
4522024-09-01T20:56:20.055ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
4532024-09-01T20:56:20.055ZINFOSledAgent (ServiceManager): No ledger in /pool/int/fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e/config/omicron-zones.json file = common/src/ledger.rs:188
4542024-09-01T20:56:20.055ZINFOSledAgent (ServiceManager): No ledger in /pool/int/c13a4209-582e-438f-b076-515144773dc5/config/omicron-zones.json file = common/src/ledger.rs:188
4552024-09-01T20:56:20.055ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
4562024-09-01T20:56:20.099ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
4572024-09-01T20:56:20.100ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
4582024-09-01T20:56:20.100ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
4592024-09-01T20:56:20.150ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_871d4f62-0910-41a9-b1f8-74b5d8646a65 file = illumos-utils/src/zone.rs:316
4602024-09-01T20:56:20.153ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_c56694eb-8314-46fd-8d68-a8cdc82c2423 file = illumos-utils/src/zone.rs:316
4612024-09-01T20:56:20.157ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_4410131d-d543-4f5e-b042-2b8f7df286aa file = illumos-utils/src/zone.rs:316
4622024-09-01T20:56:20.170ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_871d4f62-0910-41a9-b1f8-74b5d8646a65 file = illumos-utils/src/zone.rs:354
4632024-09-01T20:56:20.174ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_c56694eb-8314-46fd-8d68-a8cdc82c2423 file = illumos-utils/src/zone.rs:354
4642024-09-01T20:56:20.178ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_4410131d-d543-4f5e-b042-2b8f7df286aa file = illumos-utils/src/zone.rs:354
4652024-09-01T20:56:20.650ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 19.135208756s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
4662024-09-01T20:56:21.022ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/d40a3584-3bdb-438a-9e24-58fb6b212748): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/probe_manager.rs:230 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
4672024-09-01T20:56:21.673ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_4410131d-d543-4f5e-b042-2b8f7df286aa: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService3'/> <propval type="astring" name="gateway" value='fd00:1122:3344:1::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:1::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:1::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:1::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
4682024-09-01T20:56:21.673ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:1::, width: 64 }
4692024-09-01T20:56:22.151ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_871d4f62-0910-41a9-b1f8-74b5d8646a65: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService1'/> <propval type="astring" name="gateway" value='fd00:1122:3344:3::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:3::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:3::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:3::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
4702024-09-01T20:56:22.626ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:3::, width: 64 }
4712024-09-01T20:56:22.626ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_c56694eb-8314-46fd-8d68-a8cdc82c2423: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService2'/> <propval type="astring" name="gateway" value='fd00:1122:3344:2::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:2::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:2::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:2::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
4722024-09-01T20:56:22.626ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_4410131d-d543-4f5e-b042-2b8f7df286aa zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_4410131d-d543-4f5e-b042-2b8f7df286aa
4732024-09-01T20:56:22.629ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:2::, width: 64 }
4742024-09-01T20:56:22.629ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_871d4f62-0910-41a9-b1f8-74b5d8646a65 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_871d4f62-0910-41a9-b1f8-74b5d8646a65
4752024-09-01T20:56:22.634ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_c56694eb-8314-46fd-8d68-a8cdc82c2423 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_c56694eb-8314-46fd-8d68-a8cdc82c2423
4762024-09-01T20:56:22.757ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_871d4f62-0910-41a9-b1f8-74b5d8646a65") failed: Property not found. retry in 34.764064ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_871d4f62-0910-41a9-b1f8-74b5d8646a65
4772024-09-01T20:56:22.763ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c56694eb-8314-46fd-8d68-a8cdc82c2423") failed: Property not found. retry in 70.691004ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c56694eb-8314-46fd-8d68-a8cdc82c2423
4782024-09-01T20:56:22.769ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_4410131d-d543-4f5e-b042-2b8f7df286aa") failed: Property not found. retry in 30.837733ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_4410131d-d543-4f5e-b042-2b8f7df286aa
4792024-09-01T20:56:22.798ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_871d4f62-0910-41a9-b1f8-74b5d8646a65") failed: Property not found. retry in 94.033787ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_871d4f62-0910-41a9-b1f8-74b5d8646a65
4802024-09-01T20:56:22.807ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_4410131d-d543-4f5e-b042-2b8f7df286aa") failed: Property not found. retry in 90.172752ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_4410131d-d543-4f5e-b042-2b8f7df286aa
4812024-09-01T20:56:22.918ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c56694eb-8314-46fd-8d68-a8cdc82c2423") failed: Property not found. retry in 50.494319ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c56694eb-8314-46fd-8d68-a8cdc82c2423
4822024-09-01T20:56:22.985ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_871d4f62-0910-41a9-b1f8-74b5d8646a65") failed: Property not found. retry in 144.011077ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_871d4f62-0910-41a9-b1f8-74b5d8646a65
4832024-09-01T20:56:23.001ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_4410131d-d543-4f5e-b042-2b8f7df286aa") failed: Property not found. retry in 171.499606ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_4410131d-d543-4f5e-b042-2b8f7df286aa
4842024-09-01T20:56:23.020ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c56694eb-8314-46fd-8d68-a8cdc82c2423") failed: Property not found. retry in 290.517819ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c56694eb-8314-46fd-8d68-a8cdc82c2423
4852024-09-01T20:56:23.194ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_871d4f62-0910-41a9-b1f8-74b5d8646a65") failed: Property not found. retry in 301.634547ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_871d4f62-0910-41a9-b1f8-74b5d8646a65
4862024-09-01T20:56:23.259ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_4410131d-d543-4f5e-b042-2b8f7df286aa") failed: Property not found. retry in 471.367418ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_4410131d-d543-4f5e-b042-2b8f7df286aa
4872024-09-01T20:56:23.345ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c56694eb-8314-46fd-8d68-a8cdc82c2423") failed: Property not found. retry in 394.746761ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c56694eb-8314-46fd-8d68-a8cdc82c2423
4882024-09-01T20:56:23.509ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_871d4f62-0910-41a9-b1f8-74b5d8646a65") failed: Property not found. retry in 440.632244ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_871d4f62-0910-41a9-b1f8-74b5d8646a65
4892024-09-01T20:56:23.741ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_4410131d-d543-4f5e-b042-2b8f7df286aa") failed: Property not found. retry in 577.684225ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_4410131d-d543-4f5e-b042-2b8f7df286aa
4902024-09-01T20:56:23.748ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c56694eb-8314-46fd-8d68-a8cdc82c2423") failed: Property not found. retry in 691.292971ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c56694eb-8314-46fd-8d68-a8cdc82c2423
4912024-09-01T20:56:23.959ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_871d4f62-0910-41a9-b1f8-74b5d8646a65") failed: Property not found. retry in 910.142311ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_871d4f62-0910-41a9-b1f8-74b5d8646a65
4922024-09-01T20:56:24.328ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_4410131d-d543-4f5e-b042-2b8f7df286aa") failed: Property not found. retry in 1.015548344s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_4410131d-d543-4f5e-b042-2b8f7df286aa
4932024-09-01T20:56:24.450ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c56694eb-8314-46fd-8d68-a8cdc82c2423") failed: Property not found. retry in 805.378383ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c56694eb-8314-46fd-8d68-a8cdc82c2423
4942024-09-01T20:56:24.881ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_871d4f62-0910-41a9-b1f8-74b5d8646a65") failed: Property not found. retry in 771.481745ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_871d4f62-0910-41a9-b1f8-74b5d8646a65
4952024-09-01T20:56:25.020ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::2 exists - OK file = sled-agent/src/services.rs:4211
4962024-09-01T20:56:25.020ZINFOSledAgent (ServiceManager): Ensuring there is a default route file = sled-agent/src/services.rs:4221 gateway = fd00:1122:3344:101::1
4972024-09-01T20:56:25.027ZINFOSledAgent (ServiceManager): configuring dendrite service file = sled-agent/src/services.rs:4300
4982024-09-01T20:56:25.156ZINFOSledAgent (ServiceManager): refreshed dendrite service with new configuration file = sled-agent/src/services.rs:4348
4992024-09-01T20:56:25.156ZINFOSledAgent (ServiceManager): configuring lldp service file = sled-agent/src/services.rs:4375
5002024-09-01T20:56:25.195ZINFOSledAgent (ServiceManager): refreshed lldpd service with new configuration file = sled-agent/src/services.rs:4388
5012024-09-01T20:56:25.195ZINFOSledAgent (ServiceManager): configuring MGS service file = sled-agent/src/services.rs:4252
5022024-09-01T20:56:25.248ZINFOSledAgent (ServiceManager): refreshed MGS service with new configuration file = sled-agent/src/services.rs:4294
5032024-09-01T20:56:25.248ZINFOSledAgent (ServiceManager): configuring wicketd file = sled-agent/src/services.rs:4355 rack_subnet = fd00:1122:3344::
5042024-09-01T20:56:25.265ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c56694eb-8314-46fd-8d68-a8cdc82c2423") failed: Property not found. retry in 1.15620118s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c56694eb-8314-46fd-8d68-a8cdc82c2423
5052024-09-01T20:56:25.268ZINFOSledAgent (ServiceManager): refreshed wicketd service with new configuration file = sled-agent/src/services.rs:4366
5062024-09-01T20:56:25.268ZINFOSledAgent (ServiceManager): configuring mgd service file = sled-agent/src/services.rs:4408
5072024-09-01T20:56:25.353ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_4410131d-d543-4f5e-b042-2b8f7df286aa") failed: Property not found. retry in 939.900013ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_4410131d-d543-4f5e-b042-2b8f7df286aa
5082024-09-01T20:56:25.357ZINFOSledAgent (ServiceManager): refreshed mgd service with new configuration file = sled-agent/src/services.rs:4440
5092024-09-01T20:56:25.357ZINFOSledAgent (ServiceManager): configuring mg-ddm service file = sled-agent/src/services.rs:4446
5102024-09-01T20:56:25.466ZINFOSledAgent (ServiceManager): refreshed mg-ddm service with new configuration file = sled-agent/src/services.rs:4487
5112024-09-01T20:56:25.466ZINFOSledAgent (ServiceManager): Determining physical location of our switch zone at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:366
5122024-09-01T20:56:25.499ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/d40a3584-3bdb-438a-9e24-58fb6b212748): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/nexus.rs:571 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
5132024-09-01T20:56:25.499ZWARNSledAgent (ServiceManager): Failed to get switch ID from MGS (retrying in 48.534569ms) error = Communication Error: error sending request for url (http://[fd00:1122:3344:101::2]:12225/local/switch-id): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/bootstrap/early_networking.rs:385
5142024-09-01T20:56:25.548ZWARNSledAgent (ServiceManager): Failed to get switch ID from MGS (retrying in 66.342827ms) error = Communication Error: error sending request for url (http://[fd00:1122:3344:101::2]:12225/local/switch-id): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/bootstrap/early_networking.rs:385
5152024-09-01T20:56:25.615ZWARNSledAgent (ServiceManager): Failed to get switch ID from MGS (retrying in 211.689886ms) error = Communication Error: error sending request for url (http://[fd00:1122:3344:101::2]:12225/local/switch-id): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/bootstrap/early_networking.rs:385
5162024-09-01T20:56:25.661ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_871d4f62-0910-41a9-b1f8-74b5d8646a65") failed: Property not found. retry in 1.148246505s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_871d4f62-0910-41a9-b1f8-74b5d8646a65
5172024-09-01T20:56:25.829ZINFOSledAgent (ServiceManager): Initializing 1 Uplinks on Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:414
5182024-09-01T20:56:25.862ZINFOSledAgent (ServiceManager): Building Port Configuration file = sled-agent/src/bootstrap/early_networking.rs:676
5192024-09-01T20:56:25.862ZINFOSledAgent (ServiceManager): Checking dendrite uptime file = sled-agent/src/bootstrap/early_networking.rs:714
5202024-09-01T20:56:25.863ZINFOSledAgent (ServiceManager): Dendrite online file = sled-agent/src/bootstrap/early_networking.rs:717 uptime = 0
5212024-09-01T20:56:25.863ZINFOSledAgent (ServiceManager): Configuring default uplink on switch config = PortSettings {\n links: {\n "0": LinkSettings {\n addrs: [\n 10.151.2.174,\n ],\n params: LinkCreate {\n autoneg: false,\n fec: None,\n kr: false,\n lane: Some(\n LinkId(\n 0,\n ),\n ),\n speed: Speed40G,\n },\n },\n },\n} file = sled-agent/src/bootstrap/early_networking.rs:436
5222024-09-01T20:56:25.896ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::2]:33406
5232024-09-01T20:56:25.896ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 67 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::2]:33406 req_id = bce241fe-548e-428e-92dd-1742d3ca1065 response_code = 200 uri = /sled-identifiers
5242024-09-01T20:56:25.901ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
5252024-09-01T20:56:25.950ZINFOSledAgent (StorageManager): Received DetectedRawDisksChanged { raw_disks: {}, tx: ... } file = sled-storage/src/manager.rs:451
5262024-09-01T20:56:25.950ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
5272024-09-01T20:56:25.950ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
5282024-09-01T20:56:25.950ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
5292024-09-01T20:56:25.950ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
5302024-09-01T20:56:25.950ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
5312024-09-01T20:56:25.950ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
5322024-09-01T20:56:25.950ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
5332024-09-01T20:56:25.950ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
5342024-09-01T20:56:25.950ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
5352024-09-01T20:56:25.950ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
5362024-09-01T20:56:25.950ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
5372024-09-01T20:56:25.950ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
5382024-09-01T20:56:25.950ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
5392024-09-01T20:56:25.950ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
5402024-09-01T20:56:25.950ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
5412024-09-01T20:56:25.950ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
5422024-09-01T20:56:25.950ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
5432024-09-01T20:56:25.950ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
5442024-09-01T20:56:25.950ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
5452024-09-01T20:56:25.950ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
5462024-09-01T20:56:25.950ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
5472024-09-01T20:56:25.950ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
5482024-09-01T20:56:26.302ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_4410131d-d543-4f5e-b042-2b8f7df286aa") failed: Property not found. retry in 695.731963ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_4410131d-d543-4f5e-b042-2b8f7df286aa
5492024-09-01T20:56:26.430ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c56694eb-8314-46fd-8d68-a8cdc82c2423") failed: Property not found. retry in 1.261781523s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c56694eb-8314-46fd-8d68-a8cdc82c2423
5502024-09-01T20:56:26.819ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_871d4f62-0910-41a9-b1f8-74b5d8646a65") failed: Property not found. retry in 755.660546ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_871d4f62-0910-41a9-b1f8-74b5d8646a65
5512024-09-01T20:56:27.007ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_4410131d-d543-4f5e-b042-2b8f7df286aa") failed: Property not found. retry in 714.8073ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_4410131d-d543-4f5e-b042-2b8f7df286aa
5522024-09-01T20:56:27.584ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_871d4f62-0910-41a9-b1f8-74b5d8646a65") failed: Property not found. retry in 1.188857494s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_871d4f62-0910-41a9-b1f8-74b5d8646a65
5532024-09-01T20:56:27.701ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_c56694eb-8314-46fd-8d68-a8cdc82c2423") failed: Property not found. retry in 1.260348327s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_c56694eb-8314-46fd-8d68-a8cdc82c2423
5542024-09-01T20:56:27.733ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_4410131d-d543-4f5e-b042-2b8f7df286aa") failed: Property not found. retry in 741.047917ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_4410131d-d543-4f5e-b042-2b8f7df286aa
5552024-09-01T20:56:28.978ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_internal_dns_871d4f62-0910-41a9-b1f8-74b5d8646a65
5562024-09-01T20:56:28.978ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_internal_dns_c56694eb-8314-46fd-8d68-a8cdc82c2423
5572024-09-01T20:56:28.978ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_internal_dns_4410131d-d543-4f5e-b042-2b8f7df286aa
5582024-09-01T20:56:28.978ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
5592024-09-01T20:56:28.978ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/c13a4209-582e-438f-b076-515144773dc5/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
5602024-09-01T20:56:28.978ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 9071849 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:51413 req_id = e77b8eee-0f23-42f3-aa18-677da71fa64e response_code = 204 uri = /omicron-zones
5612024-09-01T20:56:28.980ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:1::1]:5353 file = sled-agent/src/rack_setup/service.rs:555
5622024-09-01T20:56:29.029ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:2::1]:5353 file = sled-agent/src/rack_setup/service.rs:555
5632024-09-01T20:56:29.077ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:3::1]:5353 file = sled-agent/src/rack_setup/service.rs:555
5642024-09-01T20:56:29.125ZINFOSledAgent (RSS): Configured all DNS servers file = sled-agent/src/rack_setup/service.rs:591
5652024-09-01T20:56:29.125ZINFOSledAgent (RSS): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
5662024-09-01T20:56:29.159ZINFOSledAgent (RSS): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
5672024-09-01T20:56:29.160ZINFOSledAgent (RSS): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
5682024-09-01T20:56:29.160ZINFOSledAgent (RSS): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
5692024-09-01T20:56:29.189ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(3), zones: [OmicronZoneConfig { id: 4410131d-d543-4f5e-b042-2b8f7df286aa, underlay_address: fd00:1122:3344:1::1, filesystem_pool: Some(ZpoolName { id: 067f6720-0053-47f5-be0b-12fc815674a2 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 067f6720-0053-47f5-be0b-12fc815674a2 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: c56694eb-8314-46fd-8d68-a8cdc82c2423, underlay_address: fd00:1122:3344:2::1, filesystem_pool: Some(ZpoolName { id: c65e07fa-2bd7-44d6-b309-46d959ab18fb (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: c65e07fa-2bd7-44d6-b309-46d959ab18fb (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 871d4f62-0910-41a9-b1f8-74b5d8646a65, underlay_address: fd00:1122:3344:3::1, filesystem_pool: Some(ZpoolName { id: e8cbb923-594d-495a-af32-03297bdb9d06 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: e8cbb923-594d-495a-af32-03297bdb9d06 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 85ac1227-6a8d-4cdc-a30a-2fc5ece363f8, underlay_address: fd00:1122:3344:101::1b, filesystem_pool: Some(ZpoolName { id: 6421be57-8f17-44b8-8266-2192ac87bae0 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 152ae1bb-1dfe-42d6-a186-30e49b450289, kind: Service { id: 85ac1227-6a8d-4cdc-a30a-2fc5ece363f8 }, name: Name("ntp-85ac1227-6a8d-4cdc-a30a-2fc5ece363f8"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 189, 209])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }] } file = sled-agent/src/rack_setup/service.rs:438 sled_address = [fd00:1122:3344:101::1]:12345
5702024-09-01T20:56:29.189ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:57175
5712024-09-01T20:56:29.190ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 4410131d-d543-4f5e-b042-2b8f7df286aa, dataset_name: DatasetName { pool_name: ZpoolName { id: 067f6720-0053-47f5-be0b-12fc815674a2 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
5722024-09-01T20:56:29.190ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 4410131d-d543-4f5e-b042-2b8f7df286aa, dataset_name: DatasetName { pool_name: ZpoolName { id: 067f6720-0053-47f5-be0b-12fc815674a2 (zpool), kind: External }, kind: InternalDns }, responder: ... }
5732024-09-01T20:56:29.212ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c56694eb-8314-46fd-8d68-a8cdc82c2423, dataset_name: DatasetName { pool_name: ZpoolName { id: c65e07fa-2bd7-44d6-b309-46d959ab18fb (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
5742024-09-01T20:56:29.212ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: c56694eb-8314-46fd-8d68-a8cdc82c2423, dataset_name: DatasetName { pool_name: ZpoolName { id: c65e07fa-2bd7-44d6-b309-46d959ab18fb (zpool), kind: External }, kind: InternalDns }, responder: ... }
5752024-09-01T20:56:29.231ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 871d4f62-0910-41a9-b1f8-74b5d8646a65, dataset_name: DatasetName { pool_name: ZpoolName { id: e8cbb923-594d-495a-af32-03297bdb9d06 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
5762024-09-01T20:56:29.231ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 871d4f62-0910-41a9-b1f8-74b5d8646a65, dataset_name: DatasetName { pool_name: ZpoolName { id: e8cbb923-594d-495a-af32-03297bdb9d06 (zpool), kind: External }, kind: InternalDns }, responder: ... }
5772024-09-01T20:56:29.248ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
5782024-09-01T20:56:29.248ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e/config/omicron-zones.json file = common/src/ledger.rs:177
5792024-09-01T20:56:29.248ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/c13a4209-582e-438f-b076-515144773dc5/config/omicron-zones.json file = common/src/ledger.rs:177
5802024-09-01T20:56:29.249ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
5812024-09-01T20:56:29.259ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
5822024-09-01T20:56:29.293ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
5832024-09-01T20:56:29.295ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
5842024-09-01T20:56:29.295ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
5852024-09-01T20:56:29.325ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte0", ip: 172.30.3.5, mac: MacAddr6([168, 64, 37, 255, 189, 209]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.3.1 } }) }
5862024-09-01T20:56:29.325ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1234 zone_type = boundary_ntp
5872024-09-01T20:56:29.325ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.169 file = clients/dpd-client/src/lib.rs:83
5882024-09-01T20:56:29.325ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.169 file = clients/dpd-client/src/lib.rs:90
5892024-09-01T20:56:29.326ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.169 file = clients/dpd-client/src/lib.rs:118
5902024-09-01T20:56:29.327ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
5912024-09-01T20:56:29.351ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_ntp_85ac1227-6a8d-4cdc-a30a-2fc5ece363f8 file = illumos-utils/src/zone.rs:316
5922024-09-01T20:56:29.370ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_ntp_85ac1227-6a8d-4cdc-a30a-2fc5ece363f8 file = illumos-utils/src/zone.rs:354
5932024-09-01T20:56:29.986ZINFOSledAgent (ServiceManager): Profile for oxz_ntp_85ac1227-6a8d-4cdc-a30a-2fc5ece363f8: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService4'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::1b'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/chrony-setup"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="allow" value='fd00:1122:3344:100::/56'/> <propval type="boolean" name="boundary" value='true'/> <propval type="astring" name="boundary_pool" value='boundary-ntp.control-plane.oxide.internal'/> <propval type="astring" name="server" value='0.pool.ntp.org'/> </property_group> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="1.1.1.1" /> <value_node value="9.9.9.9" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/ntp"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte0'/> <propval type="astring" name="gateway" value='172.30.3.1'/> <propval type="astring" name="ip" value='172.30.3.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
5942024-09-01T20:56:29.986ZINFOSledAgent (ServiceManager): Booting oxz_ntp_85ac1227-6a8d-4cdc-a30a-2fc5ece363f8 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_ntp_85ac1227-6a8d-4cdc-a30a-2fc5ece363f8
5952024-09-01T20:56:30.118ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_85ac1227-6a8d-4cdc-a30a-2fc5ece363f8") failed: Property not found. retry in 59.55521ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_85ac1227-6a8d-4cdc-a30a-2fc5ece363f8
5962024-09-01T20:56:30.198ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_85ac1227-6a8d-4cdc-a30a-2fc5ece363f8") failed: Property not found. retry in 51.048567ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_85ac1227-6a8d-4cdc-a30a-2fc5ece363f8
5972024-09-01T20:56:30.270ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_85ac1227-6a8d-4cdc-a30a-2fc5ece363f8") failed: Property not found. retry in 243.613578ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_85ac1227-6a8d-4cdc-a30a-2fc5ece363f8
5982024-09-01T20:56:30.524ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_85ac1227-6a8d-4cdc-a30a-2fc5ece363f8") failed: Property not found. retry in 555.453954ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_85ac1227-6a8d-4cdc-a30a-2fc5ece363f8
5992024-09-01T20:56:31.089ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_85ac1227-6a8d-4cdc-a30a-2fc5ece363f8") failed: Property not found. retry in 760.393796ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_85ac1227-6a8d-4cdc-a30a-2fc5ece363f8
6002024-09-01T20:56:31.860ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_85ac1227-6a8d-4cdc-a30a-2fc5ece363f8") failed: Property not found. retry in 1.308450414s file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_85ac1227-6a8d-4cdc-a30a-2fc5ece363f8
6012024-09-01T20:56:33.179ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_85ac1227-6a8d-4cdc-a30a-2fc5ece363f8") failed: Property not found. retry in 529.320651ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_85ac1227-6a8d-4cdc-a30a-2fc5ece363f8
6022024-09-01T20:56:33.717ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_85ac1227-6a8d-4cdc-a30a-2fc5ece363f8") failed: Property not found. retry in 597.630397ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_85ac1227-6a8d-4cdc-a30a-2fc5ece363f8
6032024-09-01T20:56:34.324ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_85ac1227-6a8d-4cdc-a30a-2fc5ece363f8") failed: Property not found. retry in 1.197714143s file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_85ac1227-6a8d-4cdc-a30a-2fc5ece363f8
6042024-09-01T20:56:35.541ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_ntp_85ac1227-6a8d-4cdc-a30a-2fc5ece363f8
6052024-09-01T20:56:35.541ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
6062024-09-01T20:56:35.541ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/c13a4209-582e-438f-b076-515144773dc5/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
6072024-09-01T20:56:35.542ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 6352010 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:57175 req_id = 94d25cd6-c2bf-4c17-b892-181d7abe68cb response_code = 204 uri = /omicron-zones
6082024-09-01T20:56:35.544ZINFOSledAgent (RSS): Waiting for rack time synchronization file = sled-agent/src/rack_setup/service.rs:632
6092024-09-01T20:56:35.574ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6102024-09-01T20:56:35.574ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:55571
6112024-09-01T20:56:35.582ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_85ac1227-6a8d-4cdc-a30a-2fc5ece363f8': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3852
6122024-09-01T20:56:35.582ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:902 latency_us = 8118 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:55571 req_id = e3b4b6b1-9bdf-410f-88ca-5e8e2af9e321 response_code = 500 uri = /timesync
6132024-09-01T20:56:35.586ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6142024-09-01T20:56:35.716ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6152024-09-01T20:56:35.716ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:63501
6162024-09-01T20:56:39.724ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_85ac1227-6a8d-4cdc-a30a-2fc5ece363f8': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3852
6172024-09-01T20:56:39.724ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:902 latency_us = 4007836 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:63501 req_id = 76b483c0-835f-44f4-864c-9c19f7d718c3 response_code = 500 uri = /timesync
6182024-09-01T20:56:39.728ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6192024-09-01T20:56:39.883ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6202024-09-01T20:56:39.883ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:42921
6212024-09-01T20:56:39.892ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_85ac1227-6a8d-4cdc-a30a-2fc5ece363f8': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3852
6222024-09-01T20:56:39.892ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:902 latency_us = 8860 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:42921 req_id = e83836fc-0438-42ae-81ff-e819d1caaf38 response_code = 500 uri = /timesync
6232024-09-01T20:56:39.895ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6242024-09-01T20:56:40.086ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6252024-09-01T20:56:40.086ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:59896
6262024-09-01T20:56:40.095ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_85ac1227-6a8d-4cdc-a30a-2fc5ece363f8': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3852
6272024-09-01T20:56:40.095ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:902 latency_us = 9176 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:59896 req_id = da17bca9-52f4-4c08-b32d-9d8e087bc311 response_code = 500 uri = /timesync
6282024-09-01T20:56:40.097ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6292024-09-01T20:56:40.299ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6302024-09-01T20:56:40.299ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:55426
6312024-09-01T20:56:45.648ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 4009162 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:55426 req_id = 39f8950a-f9ef-45b8-a05a-188f94ef6e9d response_code = 200 uri = /timesync
6322024-09-01T20:56:45.650ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: -0.416465521 } file = sled-agent/src/rack_setup/service.rs:640
6332024-09-01T20:56:45.650ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6342024-09-01T20:56:45.950ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6352024-09-01T20:56:45.950ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:49953
6362024-09-01T20:56:45.959ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8794 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:49953 req_id = 069bd331-0010-418d-a805-58d526295547 response_code = 200 uri = /timesync
6372024-09-01T20:56:45.961ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: -0.415619761 } file = sled-agent/src/rack_setup/service.rs:640
6382024-09-01T20:56:45.961ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6392024-09-01T20:56:46.244ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6402024-09-01T20:56:46.245ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:37562
6412024-09-01T20:56:46.253ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8153 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:37562 req_id = 624819ff-dccc-41ff-9fca-999b5485644b response_code = 200 uri = /timesync
6422024-09-01T20:56:46.255ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: -0.418084025 } file = sled-agent/src/rack_setup/service.rs:640
6432024-09-01T20:56:46.255ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6442024-09-01T20:56:46.452ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6452024-09-01T20:56:46.452ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:34471
6462024-09-01T20:56:46.461ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8416 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:34471 req_id = 3b5c64ae-be94-41c3-b475-c5bc8c867ba8 response_code = 200 uri = /timesync
6472024-09-01T20:56:46.463ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: -0.417518616 } file = sled-agent/src/rack_setup/service.rs:640
6482024-09-01T20:56:46.463ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6492024-09-01T20:56:47.031ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6502024-09-01T20:56:47.031ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:50201
6512024-09-01T20:56:47.039ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8019 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:50201 req_id = 1b1f8f78-d62c-48e9-bbfc-c1858f0bb7f6 response_code = 200 uri = /timesync
6522024-09-01T20:56:47.042ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: -0.419031769 } file = sled-agent/src/rack_setup/service.rs:640
6532024-09-01T20:56:47.042ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6542024-09-01T20:56:47.311ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6552024-09-01T20:56:47.312ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:58707
6562024-09-01T20:56:47.320ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8090 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:58707 req_id = 101c4b45-abfc-4b9d-9d73-af2755d1ff18 response_code = 200 uri = /timesync
6572024-09-01T20:56:47.323ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: -0.418269932 } file = sled-agent/src/rack_setup/service.rs:640
6582024-09-01T20:56:47.323ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6592024-09-01T20:56:47.906ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6602024-09-01T20:56:47.907ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:60072
6612024-09-01T20:56:47.915ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8072 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:60072 req_id = e072f215-4838-404a-8748-6ff7cfa81ab5 response_code = 200 uri = /timesync
6622024-09-01T20:56:47.917ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: -0.412792236 } file = sled-agent/src/rack_setup/service.rs:640
6632024-09-01T20:56:47.917ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6642024-09-01T20:56:48.338ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6652024-09-01T20:56:48.339ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:36885
6662024-09-01T20:56:48.347ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8242 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:36885 req_id = c58dfcbb-b390-4d6e-8f31-70125d6a3bfb response_code = 200 uri = /timesync
6672024-09-01T20:56:48.349ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: -0.41161862 } file = sled-agent/src/rack_setup/service.rs:640
6682024-09-01T20:56:48.349ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6692024-09-01T20:56:48.889ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6702024-09-01T20:56:48.889ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:48807
6712024-09-01T20:56:48.897ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8045 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:48807 req_id = 1242b1e0-2916-483f-a9c7-788d41a3e5a2 response_code = 200 uri = /timesync
6722024-09-01T20:56:48.901ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: -0.410150498 } file = sled-agent/src/rack_setup/service.rs:640
6732024-09-01T20:56:48.901ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6742024-09-01T20:56:49.723ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6752024-09-01T20:56:49.723ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:63343
6762024-09-01T20:56:49.731ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8519 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:63343 req_id = f182de32-a969-4a6a-858d-2edd467212c2 response_code = 200 uri = /timesync
6772024-09-01T20:56:49.735ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2728380539, ip_addr: 162.159.200.123, stratum: 4, ref_time: 1725224208.6463687, correction: -0.408047497 } file = sled-agent/src/rack_setup/service.rs:640
6782024-09-01T20:56:49.735ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6792024-09-01T20:56:50.452ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6802024-09-01T20:56:50.452ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:48144
6812024-09-01T20:56:50.461ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8320 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:48144 req_id = 369433ed-5eaf-4228-9498-e641cb736822 response_code = 200 uri = /timesync
6822024-09-01T20:56:50.463ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2728380539, ip_addr: 162.159.200.123, stratum: 4, ref_time: 1725224208.6463687, correction: -0.406066924 } file = sled-agent/src/rack_setup/service.rs:640
6832024-09-01T20:56:50.463ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6842024-09-01T20:56:51.660ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6852024-09-01T20:56:51.660ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:64790
6862024-09-01T20:56:51.670ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 9176 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:64790 req_id = 62ef2e1b-66e1-4c2b-8b5a-184353ac6a84 response_code = 200 uri = /timesync
6872024-09-01T20:56:51.672ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2728380539, ip_addr: 162.159.200.123, stratum: 4, ref_time: 1725224211.161549, correction: -0.402627289 } file = sled-agent/src/rack_setup/service.rs:640
6882024-09-01T20:56:51.672ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6892024-09-01T20:56:53.642ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6902024-09-01T20:56:53.642ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:62267
6912024-09-01T20:56:53.650ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8261 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:62267 req_id = 89cb6b71-4a4c-40ab-8d04-c2ed77d092e3 response_code = 200 uri = /timesync
6922024-09-01T20:56:53.654ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2728380539, ip_addr: 162.159.200.123, stratum: 4, ref_time: 1725224211.4869063, correction: -0.398444623 } file = sled-agent/src/rack_setup/service.rs:640
6932024-09-01T20:56:53.654ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6942024-09-01T20:56:56.086ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6952024-09-01T20:56:56.086ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:41979
6962024-09-01T20:56:56.096ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 9703 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:41979 req_id = fac5bad8-476e-4248-b8c3-6d97bca41d82 response_code = 200 uri = /timesync
6972024-09-01T20:56:56.099ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2728380539, ip_addr: 162.159.200.123, stratum: 4, ref_time: 1725224214.752572, correction: -0.392407268 } file = sled-agent/src/rack_setup/service.rs:640
6982024-09-01T20:56:56.099ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6992024-09-01T20:56:56.133ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 163.570447ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
7002024-09-01T20:56:56.533ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:43991
7012024-09-01T20:56:56.533ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 71 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:43991 req_id = 4bea2f25-0c5d-409c-b0f1-8d201fcfabd2 response_code = 200 uri = /baseboard
7022024-09-01T20:56:57.466ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
7032024-09-01T20:56:57.466ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:40265
7042024-09-01T20:56:57.475ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8606 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:40265 req_id = 2383a921-4de0-47cf-af0e-7695941b633c response_code = 200 uri = /timesync
7052024-09-01T20:56:57.478ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2728380539, ip_addr: 162.159.200.123, stratum: 4, ref_time: 1725224216.3038874, correction: -0.388643622 } file = sled-agent/src/rack_setup/service.rs:640
7062024-09-01T20:56:57.478ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
7072024-09-01T20:56:58.867ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
7082024-09-01T20:56:58.868ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:42522
7092024-09-01T20:56:58.876ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8689 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:42522 req_id = bbe9e0a2-92ad-4e52-b9bb-e3d27a758178 response_code = 200 uri = /timesync
7102024-09-01T20:56:58.880ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2728380539, ip_addr: 162.159.200.123, stratum: 4, ref_time: 1725224217.7407336, correction: -0.38468349 } file = sled-agent/src/rack_setup/service.rs:640
7112024-09-01T20:56:58.880ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
7122024-09-01T20:57:03.079ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
7132024-09-01T20:57:03.079ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:55153
7142024-09-01T20:57:03.088ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 9152 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:55153 req_id = 643e4702-e0fb-4bb1-8413-c257f0e633a5 response_code = 200 uri = /timesync
7152024-09-01T20:57:03.091ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2728380539, ip_addr: 162.159.200.123, stratum: 4, ref_time: 1725224221.8687384, correction: -0.373134762 } file = sled-agent/src/rack_setup/service.rs:640
7162024-09-01T20:57:03.091ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
7172024-09-01T20:57:08.683ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
7182024-09-01T20:57:08.683ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:36809
7192024-09-01T20:57:08.692ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8974 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:36809 req_id = 259f322e-e24c-46e3-8f98-76598b3cd93f response_code = 200 uri = /timesync
7202024-09-01T20:57:08.695ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2728380539, ip_addr: 162.159.200.123, stratum: 4, ref_time: 1725224226.505738, correction: -0.358425558 } file = sled-agent/src/rack_setup/service.rs:640
7212024-09-01T20:57:08.695ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
7222024-09-01T20:57:11.292ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 742.345914ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
7232024-09-01T20:57:15.441ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
7242024-09-01T20:57:15.442ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:58235
7252024-09-01T20:57:15.450ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8218 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:58235 req_id = 6b8e9865-a67a-4e9b-8f7b-a2b2414cec2b response_code = 200 uri = /timesync
7262024-09-01T20:57:15.453ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2728380539, ip_addr: 162.159.200.123, stratum: 4, ref_time: 1725224233.3132737, correction: -0.342090368 } file = sled-agent/src/rack_setup/service.rs:640
7272024-09-01T20:57:15.453ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
7282024-09-01T20:57:22.732ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
7292024-09-01T20:57:22.732ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:54081
7302024-09-01T20:57:22.741ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8341 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:54081 req_id = 041bf2fd-dc99-46b6-a58c-10fb0c5ab41e response_code = 200 uri = /timesync
7312024-09-01T20:57:22.745ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2728380539, ip_addr: 162.159.200.123, stratum: 4, ref_time: 1725224240.78455, correction: -0.322165072 } file = sled-agent/src/rack_setup/service.rs:640
7322024-09-01T20:57:22.745ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
7332024-09-01T20:57:26.486ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:47551
7342024-09-01T20:57:26.486ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 44 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:47551 req_id = bbb76e82-3536-4b8a-9b85-ef4bbfaf7699 response_code = 200 uri = /baseboard
7352024-09-01T20:57:27.026ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 1.041914212s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
7362024-09-01T20:57:31.577ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
7372024-09-01T20:57:31.577ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:56409
7382024-09-01T20:57:31.587ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 9693 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:56409 req_id = 46104b49-8b53-4203-b19c-6f33aed64cb7 response_code = 200 uri = /timesync
7392024-09-01T20:57:31.589ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2728380539, ip_addr: 162.159.200.123, stratum: 4, ref_time: 1725224249.2655685, correction: -0.298218638 } file = sled-agent/src/rack_setup/service.rs:640
7402024-09-01T20:57:31.589ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
7412024-09-01T20:57:40.866ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
7422024-09-01T20:57:40.866ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:37989
7432024-09-01T20:57:40.875ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 9130 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:37989 req_id = faafdd29-e791-4942-85d3-b3adfca62f18 response_code = 200 uri = /timesync
7442024-09-01T20:57:40.878ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2728380539, ip_addr: 162.159.200.123, stratum: 4, ref_time: 1725224253.0620208, correction: -0.272860974 } file = sled-agent/src/rack_setup/service.rs:640
7452024-09-01T20:57:40.878ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
7462024-09-01T20:57:43.061ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 2.676038204s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
7472024-09-01T20:57:51.083ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
7482024-09-01T20:57:51.083ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:42117
7492024-09-01T20:57:51.093ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 9596 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:42117 req_id = ea3cfd12-2d58-45ca-9248-98f6076f6564 response_code = 200 uri = /timesync
7502024-09-01T20:57:51.095ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2728380539, ip_addr: 162.159.200.123, stratum: 4, ref_time: 1725224269.9169064, correction: -0.245124266 } file = sled-agent/src/rack_setup/service.rs:640
7512024-09-01T20:57:51.095ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
7522024-09-01T20:57:56.439ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:39959
7532024-09-01T20:57:56.439ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 623 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:39959 req_id = 5a6b3ad5-6bf2-4165-9af6-bda21192698c response_code = 200 uri = /baseboard
7542024-09-01T20:58:00.724ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 2.971859104s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
7552024-09-01T20:58:02.915ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
7562024-09-01T20:58:02.915ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:39477
7572024-09-01T20:58:02.924ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8959 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:39477 req_id = 000dcbc4-cdb3-4015-8eea-4da2237b0181 response_code = 200 uri = /timesync
7582024-09-01T20:58:02.927ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2728380539, ip_addr: 162.159.200.123, stratum: 4, ref_time: 1725224277.516203, correction: -0.21300143 } file = sled-agent/src/rack_setup/service.rs:640
7592024-09-01T20:58:02.927ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
7602024-09-01T20:58:18.683ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 5.251900049s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
7612024-09-01T20:58:20.380ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
7622024-09-01T20:58:20.380ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:47351
7632024-09-01T20:58:20.389ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8697 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:47351 req_id = 330dd7ad-f066-4442-9045-ab46d250029a response_code = 200 uri = /timesync
7642024-09-01T20:58:20.394ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2728380539, ip_addr: 162.159.200.123, stratum: 4, ref_time: 1725224294.238478, correction: -0.165575996 } file = sled-agent/src/rack_setup/service.rs:640
7652024-09-01T20:58:20.394ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
7662024-09-01T20:58:26.390ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:33926
7672024-09-01T20:58:26.390ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 34 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:33926 req_id = 2ef34b18-eee6-4477-9da1-a1091eb66336 response_code = 200 uri = /baseboard
7682024-09-01T20:58:38.915ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 18.178614105s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
7692024-09-01T20:58:40.624ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
7702024-09-01T20:58:40.624ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:40677
7712024-09-01T20:58:40.633ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8860 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:40677 req_id = e332e643-93cc-4a29-8c49-7ace9d1194d2 response_code = 200 uri = /timesync
7722024-09-01T20:58:40.637ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 3494642188, ip_addr: 208.76.2.12, stratum: 2, ref_time: 1725224320.047177, correction: -0.112490661 } file = sled-agent/src/rack_setup/service.rs:640
7732024-09-01T20:58:40.637ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
7742024-09-01T20:58:56.343ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:48903
7752024-09-01T20:58:56.343ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 47 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:48903 req_id = fcfd9495-e473-429e-85b4-af30a2af3ac6 response_code = 200 uri = /baseboard
7762024-09-01T20:59:09.616ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
7772024-09-01T20:59:09.616ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:43745
7782024-09-01T20:59:09.637ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 20700 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:43745 req_id = 400f9ec8-b94c-4ff9-97d0-21f06979f384 response_code = 200 uri = /timesync
7792024-09-01T20:59:09.639ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: true, ref_id: 3494642188, ip_addr: 208.76.2.12, stratum: 2, ref_time: 1725224347.0663536, correction: -0.032345165 } file = sled-agent/src/rack_setup/service.rs:640
7802024-09-01T20:59:09.639ZINFOSledAgent (RSS): Finished setting up Internal DNS and NTP file = sled-agent/src/rack_setup/service.rs:1266
7812024-09-01T20:59:09.668ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 4410131d-d543-4f5e-b042-2b8f7df286aa, underlay_address: fd00:1122:3344:1::1, filesystem_pool: Some(ZpoolName { id: 067f6720-0053-47f5-be0b-12fc815674a2 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 067f6720-0053-47f5-be0b-12fc815674a2 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: c56694eb-8314-46fd-8d68-a8cdc82c2423, underlay_address: fd00:1122:3344:2::1, filesystem_pool: Some(ZpoolName { id: c65e07fa-2bd7-44d6-b309-46d959ab18fb (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: c65e07fa-2bd7-44d6-b309-46d959ab18fb (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 871d4f62-0910-41a9-b1f8-74b5d8646a65, underlay_address: fd00:1122:3344:3::1, filesystem_pool: Some(ZpoolName { id: e8cbb923-594d-495a-af32-03297bdb9d06 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: e8cbb923-594d-495a-af32-03297bdb9d06 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 85ac1227-6a8d-4cdc-a30a-2fc5ece363f8, underlay_address: fd00:1122:3344:101::1b, filesystem_pool: Some(ZpoolName { id: 6421be57-8f17-44b8-8266-2192ac87bae0 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 152ae1bb-1dfe-42d6-a186-30e49b450289, kind: Service { id: 85ac1227-6a8d-4cdc-a30a-2fc5ece363f8 }, name: Name("ntp-85ac1227-6a8d-4cdc-a30a-2fc5ece363f8"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 189, 209])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: b3dc62d6-da69-4341-a243-bce80bbf280e, underlay_address: fd00:1122:3344:101::3, filesystem_pool: Some(ZpoolName { id: 067f6720-0053-47f5-be0b-12fc815674a2 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 067f6720-0053-47f5-be0b-12fc815674a2 (zpool), kind: External } } } }, OmicronZoneConfig { id: 94377199-0f3e-43d9-a00d-b82f88d48ad6, underlay_address: fd00:1122:3344:101::4, filesystem_pool: Some(ZpoolName { id: c65e07fa-2bd7-44d6-b309-46d959ab18fb (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: c65e07fa-2bd7-44d6-b309-46d959ab18fb (zpool), kind: External } } } }, OmicronZoneConfig { id: 0894f236-b210-4f67-b3d2-170d63897b72, underlay_address: fd00:1122:3344:101::5, filesystem_pool: Some(ZpoolName { id: e8cbb923-594d-495a-af32-03297bdb9d06 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: e8cbb923-594d-495a-af32-03297bdb9d06 (zpool), kind: External } } } }, OmicronZoneConfig { id: 25121651-084b-4db4-8053-a44075741d18, underlay_address: fd00:1122:3344:101::6, filesystem_pool: Some(ZpoolName { id: b7b09d01-4b5a-46f4-a346-2e7b6a0e3462 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: b7b09d01-4b5a-46f4-a346-2e7b6a0e3462 (zpool), kind: External } } } }, OmicronZoneConfig { id: 396f2f68-ce7b-4018-8f2d-7aaea21e47d6, underlay_address: fd00:1122:3344:101::7, filesystem_pool: Some(ZpoolName { id: 14681c40-9950-4e30-8f61-ec7a2ca60e1c (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 14681c40-9950-4e30-8f61-ec7a2ca60e1c (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:438 sled_address = [fd00:1122:3344:101::1]:12345
7822024-09-01T20:59:09.668ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:43832
7832024-09-01T20:59:09.669ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 4410131d-d543-4f5e-b042-2b8f7df286aa, dataset_name: DatasetName { pool_name: ZpoolName { id: 067f6720-0053-47f5-be0b-12fc815674a2 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
7842024-09-01T20:59:09.669ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 4410131d-d543-4f5e-b042-2b8f7df286aa, dataset_name: DatasetName { pool_name: ZpoolName { id: 067f6720-0053-47f5-be0b-12fc815674a2 (zpool), kind: External }, kind: InternalDns }, responder: ... }
7852024-09-01T20:59:09.690ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c56694eb-8314-46fd-8d68-a8cdc82c2423, dataset_name: DatasetName { pool_name: ZpoolName { id: c65e07fa-2bd7-44d6-b309-46d959ab18fb (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
7862024-09-01T20:59:09.690ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: c56694eb-8314-46fd-8d68-a8cdc82c2423, dataset_name: DatasetName { pool_name: ZpoolName { id: c65e07fa-2bd7-44d6-b309-46d959ab18fb (zpool), kind: External }, kind: InternalDns }, responder: ... }
7872024-09-01T20:59:09.709ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 871d4f62-0910-41a9-b1f8-74b5d8646a65, dataset_name: DatasetName { pool_name: ZpoolName { id: e8cbb923-594d-495a-af32-03297bdb9d06 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
7882024-09-01T20:59:09.709ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 871d4f62-0910-41a9-b1f8-74b5d8646a65, dataset_name: DatasetName { pool_name: ZpoolName { id: e8cbb923-594d-495a-af32-03297bdb9d06 (zpool), kind: External }, kind: InternalDns }, responder: ... }
7892024-09-01T20:59:09.727ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: b3dc62d6-da69-4341-a243-bce80bbf280e, dataset_name: DatasetName { pool_name: ZpoolName { id: 067f6720-0053-47f5-be0b-12fc815674a2 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
7902024-09-01T20:59:09.727ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: b3dc62d6-da69-4341-a243-bce80bbf280e, dataset_name: DatasetName { pool_name: ZpoolName { id: 067f6720-0053-47f5-be0b-12fc815674a2 (zpool), kind: External }, kind: Cockroach }, responder: ... }
7912024-09-01T20:59:09.778ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 94377199-0f3e-43d9-a00d-b82f88d48ad6, dataset_name: DatasetName { pool_name: ZpoolName { id: c65e07fa-2bd7-44d6-b309-46d959ab18fb (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
7922024-09-01T20:59:09.778ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 94377199-0f3e-43d9-a00d-b82f88d48ad6, dataset_name: DatasetName { pool_name: ZpoolName { id: c65e07fa-2bd7-44d6-b309-46d959ab18fb (zpool), kind: External }, kind: Cockroach }, responder: ... }
7932024-09-01T20:59:09.832ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 0894f236-b210-4f67-b3d2-170d63897b72, dataset_name: DatasetName { pool_name: ZpoolName { id: e8cbb923-594d-495a-af32-03297bdb9d06 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
7942024-09-01T20:59:09.832ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 0894f236-b210-4f67-b3d2-170d63897b72, dataset_name: DatasetName { pool_name: ZpoolName { id: e8cbb923-594d-495a-af32-03297bdb9d06 (zpool), kind: External }, kind: Cockroach }, responder: ... }
7952024-09-01T20:59:09.883ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 25121651-084b-4db4-8053-a44075741d18, dataset_name: DatasetName { pool_name: ZpoolName { id: b7b09d01-4b5a-46f4-a346-2e7b6a0e3462 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
7962024-09-01T20:59:09.883ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 25121651-084b-4db4-8053-a44075741d18, dataset_name: DatasetName { pool_name: ZpoolName { id: b7b09d01-4b5a-46f4-a346-2e7b6a0e3462 (zpool), kind: External }, kind: Cockroach }, responder: ... }
7972024-09-01T20:59:09.935ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 396f2f68-ce7b-4018-8f2d-7aaea21e47d6, dataset_name: DatasetName { pool_name: ZpoolName { id: 14681c40-9950-4e30-8f61-ec7a2ca60e1c (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
7982024-09-01T20:59:09.935ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 396f2f68-ce7b-4018-8f2d-7aaea21e47d6, dataset_name: DatasetName { pool_name: ZpoolName { id: 14681c40-9950-4e30-8f61-ec7a2ca60e1c (zpool), kind: External }, kind: Cockroach }, responder: ... }
7992024-09-01T20:59:09.987ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8002024-09-01T20:59:09.987ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e/config/omicron-zones.json file = common/src/ledger.rs:177
8012024-09-01T20:59:09.987ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/c13a4209-582e-438f-b076-515144773dc5/config/omicron-zones.json file = common/src/ledger.rs:177
8022024-09-01T20:59:09.987ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8032024-09-01T20:59:10.071ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8042024-09-01T20:59:10.071ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8052024-09-01T20:59:10.071ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8062024-09-01T20:59:10.071ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8072024-09-01T20:59:10.071ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8082024-09-01T20:59:10.161ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_396f2f68-ce7b-4018-8f2d-7aaea21e47d6 file = illumos-utils/src/zone.rs:316
8092024-09-01T20:59:10.165ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_94377199-0f3e-43d9-a00d-b82f88d48ad6 file = illumos-utils/src/zone.rs:316
8102024-09-01T20:59:10.168ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_b3dc62d6-da69-4341-a243-bce80bbf280e file = illumos-utils/src/zone.rs:316
8112024-09-01T20:59:10.172ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_25121651-084b-4db4-8053-a44075741d18 file = illumos-utils/src/zone.rs:316
8122024-09-01T20:59:10.177ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_0894f236-b210-4f67-b3d2-170d63897b72 file = illumos-utils/src/zone.rs:316
8132024-09-01T20:59:10.182ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_396f2f68-ce7b-4018-8f2d-7aaea21e47d6 file = illumos-utils/src/zone.rs:354
8142024-09-01T20:59:10.186ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_94377199-0f3e-43d9-a00d-b82f88d48ad6 file = illumos-utils/src/zone.rs:354
8152024-09-01T20:59:10.190ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_b3dc62d6-da69-4341-a243-bce80bbf280e file = illumos-utils/src/zone.rs:354
8162024-09-01T20:59:10.194ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_25121651-084b-4db4-8053-a44075741d18 file = illumos-utils/src/zone.rs:354
8172024-09-01T20:59:10.199ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_0894f236-b210-4f67-b3d2-170d63897b72 file = illumos-utils/src/zone.rs:354
8182024-09-01T20:59:12.042ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 32.612181003s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
8192024-09-01T20:59:13.072ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_396f2f68-ce7b-4018-8f2d-7aaea21e47d6: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService5'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::7'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::7]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='396f2f68-ce7b-4018-8f2d-7aaea21e47d6'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::7]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::7]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8202024-09-01T20:59:13.072ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_396f2f68-ce7b-4018-8f2d-7aaea21e47d6 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_396f2f68-ce7b-4018-8f2d-7aaea21e47d6
8212024-09-01T20:59:13.115ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_0894f236-b210-4f67-b3d2-170d63897b72: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService9'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::5]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='0894f236-b210-4f67-b3d2-170d63897b72'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::5]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::5]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8222024-09-01T20:59:13.115ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_0894f236-b210-4f67-b3d2-170d63897b72 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_0894f236-b210-4f67-b3d2-170d63897b72
8232024-09-01T20:59:13.151ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_b3dc62d6-da69-4341-a243-bce80bbf280e: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService7'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::3'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::3]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='b3dc62d6-da69-4341-a243-bce80bbf280e'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::3]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::3]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8242024-09-01T20:59:13.151ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_b3dc62d6-da69-4341-a243-bce80bbf280e zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_b3dc62d6-da69-4341-a243-bce80bbf280e
8252024-09-01T20:59:13.157ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_94377199-0f3e-43d9-a00d-b82f88d48ad6: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService6'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::4'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::4]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='94377199-0f3e-43d9-a00d-b82f88d48ad6'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::4]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::4]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8262024-09-01T20:59:13.157ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_94377199-0f3e-43d9-a00d-b82f88d48ad6 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_94377199-0f3e-43d9-a00d-b82f88d48ad6
8272024-09-01T20:59:13.221ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_396f2f68-ce7b-4018-8f2d-7aaea21e47d6") failed: Property not found. retry in 61.074109ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_396f2f68-ce7b-4018-8f2d-7aaea21e47d6
8282024-09-01T20:59:13.225ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_25121651-084b-4db4-8053-a44075741d18: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService8'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::6]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='25121651-084b-4db4-8053-a44075741d18'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::6]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::6]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8292024-09-01T20:59:13.225ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_25121651-084b-4db4-8053-a44075741d18 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_25121651-084b-4db4-8053-a44075741d18
8302024-09-01T20:59:13.254ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0894f236-b210-4f67-b3d2-170d63897b72") failed: Property not found. retry in 44.367391ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0894f236-b210-4f67-b3d2-170d63897b72
8312024-09-01T20:59:13.311ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_396f2f68-ce7b-4018-8f2d-7aaea21e47d6") failed: Property not found. retry in 71.319752ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_396f2f68-ce7b-4018-8f2d-7aaea21e47d6
8322024-09-01T20:59:13.329ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_94377199-0f3e-43d9-a00d-b82f88d48ad6") failed: Property not found. retry in 28.562135ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_94377199-0f3e-43d9-a00d-b82f88d48ad6
8332024-09-01T20:59:13.434ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b3dc62d6-da69-4341-a243-bce80bbf280e") failed: Property not found. retry in 71.73932ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b3dc62d6-da69-4341-a243-bce80bbf280e
8342024-09-01T20:59:13.457ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0894f236-b210-4f67-b3d2-170d63897b72") failed: Property not found. retry in 71.480361ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0894f236-b210-4f67-b3d2-170d63897b72
8352024-09-01T20:59:13.474ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_396f2f68-ce7b-4018-8f2d-7aaea21e47d6") failed: Property not found. retry in 198.445759ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_396f2f68-ce7b-4018-8f2d-7aaea21e47d6
8362024-09-01T20:59:13.618ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_94377199-0f3e-43d9-a00d-b82f88d48ad6") failed: Property not found. retry in 137.500681ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_94377199-0f3e-43d9-a00d-b82f88d48ad6
8372024-09-01T20:59:13.678ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b3dc62d6-da69-4341-a243-bce80bbf280e") failed: Property not found. retry in 142.979918ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b3dc62d6-da69-4341-a243-bce80bbf280e
8382024-09-01T20:59:13.833ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0894f236-b210-4f67-b3d2-170d63897b72") failed: Property not found. retry in 283.369361ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0894f236-b210-4f67-b3d2-170d63897b72
8392024-09-01T20:59:14.052ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_396f2f68-ce7b-4018-8f2d-7aaea21e47d6") failed: Property not found. retry in 239.886583ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_396f2f68-ce7b-4018-8f2d-7aaea21e47d6
8402024-09-01T20:59:14.334ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_94377199-0f3e-43d9-a00d-b82f88d48ad6") failed: Property not found. retry in 101.579087ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_94377199-0f3e-43d9-a00d-b82f88d48ad6
8412024-09-01T20:59:14.537ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b3dc62d6-da69-4341-a243-bce80bbf280e") failed: Property not found. retry in 191.656817ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b3dc62d6-da69-4341-a243-bce80bbf280e
8422024-09-01T20:59:14.714ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_25121651-084b-4db4-8053-a44075741d18") failed: Property not found. retry in 68.843882ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_25121651-084b-4db4-8053-a44075741d18
8432024-09-01T20:59:14.785ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0894f236-b210-4f67-b3d2-170d63897b72") failed: Property not found. retry in 505.504582ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0894f236-b210-4f67-b3d2-170d63897b72
8442024-09-01T20:59:14.920ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_396f2f68-ce7b-4018-8f2d-7aaea21e47d6") failed: Property not found. retry in 537.630991ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_396f2f68-ce7b-4018-8f2d-7aaea21e47d6
8452024-09-01T20:59:14.950ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_94377199-0f3e-43d9-a00d-b82f88d48ad6") failed: Property not found. retry in 262.914733ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_94377199-0f3e-43d9-a00d-b82f88d48ad6
8462024-09-01T20:59:15.006ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b3dc62d6-da69-4341-a243-bce80bbf280e") failed: Property not found. retry in 429.111853ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b3dc62d6-da69-4341-a243-bce80bbf280e
8472024-09-01T20:59:15.266ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_25121651-084b-4db4-8053-a44075741d18") failed: Property not found. retry in 97.757167ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_25121651-084b-4db4-8053-a44075741d18
8482024-09-01T20:59:15.377ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_94377199-0f3e-43d9-a00d-b82f88d48ad6") failed: Property not found. retry in 963.989022ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_94377199-0f3e-43d9-a00d-b82f88d48ad6
8492024-09-01T20:59:15.436ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_25121651-084b-4db4-8053-a44075741d18") failed: Property not found. retry in 113.996795ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_25121651-084b-4db4-8053-a44075741d18
8502024-09-01T20:59:15.450ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0894f236-b210-4f67-b3d2-170d63897b72") failed: Property not found. retry in 1.028989153s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0894f236-b210-4f67-b3d2-170d63897b72
8512024-09-01T20:59:15.464ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_396f2f68-ce7b-4018-8f2d-7aaea21e47d6") failed: Property not found. retry in 991.737365ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_396f2f68-ce7b-4018-8f2d-7aaea21e47d6
8522024-09-01T20:59:15.480ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b3dc62d6-da69-4341-a243-bce80bbf280e") failed: Property not found. retry in 400.713174ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b3dc62d6-da69-4341-a243-bce80bbf280e
8532024-09-01T20:59:15.561ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_25121651-084b-4db4-8053-a44075741d18") failed: Property not found. retry in 478.181214ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_25121651-084b-4db4-8053-a44075741d18
8542024-09-01T20:59:15.890ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b3dc62d6-da69-4341-a243-bce80bbf280e") failed: Property not found. retry in 1.488583609s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b3dc62d6-da69-4341-a243-bce80bbf280e
8552024-09-01T20:59:16.050ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_25121651-084b-4db4-8053-a44075741d18") failed: Property not found. retry in 579.604164ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_25121651-084b-4db4-8053-a44075741d18
8562024-09-01T20:59:16.348ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_94377199-0f3e-43d9-a00d-b82f88d48ad6") failed: Property not found. retry in 850.422232ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_94377199-0f3e-43d9-a00d-b82f88d48ad6
8572024-09-01T20:59:16.462ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_396f2f68-ce7b-4018-8f2d-7aaea21e47d6") failed: Property not found. retry in 1.39277761s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_396f2f68-ce7b-4018-8f2d-7aaea21e47d6
8582024-09-01T20:59:16.484ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0894f236-b210-4f67-b3d2-170d63897b72") failed: Property not found. retry in 1.063693878s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0894f236-b210-4f67-b3d2-170d63897b72
8592024-09-01T20:59:16.637ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_25121651-084b-4db4-8053-a44075741d18") failed: Property not found. retry in 584.712384ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_25121651-084b-4db4-8053-a44075741d18
8602024-09-01T20:59:17.207ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_94377199-0f3e-43d9-a00d-b82f88d48ad6") failed: Property not found. retry in 942.507845ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_94377199-0f3e-43d9-a00d-b82f88d48ad6
8612024-09-01T20:59:17.230ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_25121651-084b-4db4-8053-a44075741d18") failed: Property not found. retry in 1.380794775s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_25121651-084b-4db4-8053-a44075741d18
8622024-09-01T20:59:17.384ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b3dc62d6-da69-4341-a243-bce80bbf280e") failed: Property not found. retry in 800.635844ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b3dc62d6-da69-4341-a243-bce80bbf280e
8632024-09-01T20:59:17.555ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0894f236-b210-4f67-b3d2-170d63897b72") failed: Property not found. retry in 676.017205ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0894f236-b210-4f67-b3d2-170d63897b72
8642024-09-01T20:59:17.860ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_396f2f68-ce7b-4018-8f2d-7aaea21e47d6") failed: Property not found. retry in 909.964368ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_396f2f68-ce7b-4018-8f2d-7aaea21e47d6
8652024-09-01T20:59:18.154ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_94377199-0f3e-43d9-a00d-b82f88d48ad6") failed: Property not found. retry in 791.32263ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_94377199-0f3e-43d9-a00d-b82f88d48ad6
8662024-09-01T20:59:18.192ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b3dc62d6-da69-4341-a243-bce80bbf280e") failed: Property not found. retry in 1.133554154s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b3dc62d6-da69-4341-a243-bce80bbf280e
8672024-09-01T20:59:18.238ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0894f236-b210-4f67-b3d2-170d63897b72") failed: Property not found. retry in 1.080361554s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0894f236-b210-4f67-b3d2-170d63897b72
8682024-09-01T20:59:18.616ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_25121651-084b-4db4-8053-a44075741d18") failed: Property not found. retry in 610.923001ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_25121651-084b-4db4-8053-a44075741d18
8692024-09-01T20:59:18.793ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_396f2f68-ce7b-4018-8f2d-7aaea21e47d6") failed: Property not found. retry in 714.040141ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_396f2f68-ce7b-4018-8f2d-7aaea21e47d6
8702024-09-01T20:59:18.953ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_94377199-0f3e-43d9-a00d-b82f88d48ad6") failed: Property not found. retry in 943.650268ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_94377199-0f3e-43d9-a00d-b82f88d48ad6
8712024-09-01T20:59:19.236ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_25121651-084b-4db4-8053-a44075741d18") failed: Property not found. retry in 673.362227ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_25121651-084b-4db4-8053-a44075741d18
8722024-09-01T20:59:19.325ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0894f236-b210-4f67-b3d2-170d63897b72") failed: Property not found. retry in 1.193199519s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0894f236-b210-4f67-b3d2-170d63897b72
8732024-09-01T20:59:19.333ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b3dc62d6-da69-4341-a243-bce80bbf280e") failed: Property not found. retry in 1.295103295s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b3dc62d6-da69-4341-a243-bce80bbf280e
8742024-09-01T20:59:19.514ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_396f2f68-ce7b-4018-8f2d-7aaea21e47d6") failed: Property not found. retry in 663.389198ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_396f2f68-ce7b-4018-8f2d-7aaea21e47d6
8752024-09-01T20:59:19.904ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_94377199-0f3e-43d9-a00d-b82f88d48ad6") failed: Property not found. retry in 551.641259ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_94377199-0f3e-43d9-a00d-b82f88d48ad6
8762024-09-01T20:59:19.917ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_25121651-084b-4db4-8053-a44075741d18") failed: Property not found. retry in 613.011589ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_25121651-084b-4db4-8053-a44075741d18
8772024-09-01T20:59:20.186ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_396f2f68-ce7b-4018-8f2d-7aaea21e47d6") failed: Property not found. retry in 754.134753ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_396f2f68-ce7b-4018-8f2d-7aaea21e47d6
8782024-09-01T20:59:20.464ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_94377199-0f3e-43d9-a00d-b82f88d48ad6") failed: Property not found. retry in 1.052698868s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_94377199-0f3e-43d9-a00d-b82f88d48ad6
8792024-09-01T20:59:20.526ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0894f236-b210-4f67-b3d2-170d63897b72") failed: Property not found. retry in 1.204334483s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0894f236-b210-4f67-b3d2-170d63897b72
8802024-09-01T20:59:20.537ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_25121651-084b-4db4-8053-a44075741d18") failed: Property not found. retry in 541.643082ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_25121651-084b-4db4-8053-a44075741d18
8812024-09-01T20:59:20.635ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b3dc62d6-da69-4341-a243-bce80bbf280e") failed: Property not found. retry in 1.303525507s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b3dc62d6-da69-4341-a243-bce80bbf280e
8822024-09-01T20:59:20.947ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_396f2f68-ce7b-4018-8f2d-7aaea21e47d6") failed: Property not found. retry in 983.772783ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_396f2f68-ce7b-4018-8f2d-7aaea21e47d6
8832024-09-01T20:59:21.087ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_25121651-084b-4db4-8053-a44075741d18") failed: Property not found. retry in 677.466107ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_25121651-084b-4db4-8053-a44075741d18
8842024-09-01T20:59:21.525ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_94377199-0f3e-43d9-a00d-b82f88d48ad6") failed: Property not found. retry in 803.852963ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_94377199-0f3e-43d9-a00d-b82f88d48ad6
8852024-09-01T20:59:21.736ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0894f236-b210-4f67-b3d2-170d63897b72") failed: Property not found. retry in 843.510489ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0894f236-b210-4f67-b3d2-170d63897b72
8862024-09-01T20:59:21.771ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_25121651-084b-4db4-8053-a44075741d18") failed: Property not found. retry in 756.928107ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_25121651-084b-4db4-8053-a44075741d18
8872024-09-01T20:59:21.937ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_396f2f68-ce7b-4018-8f2d-7aaea21e47d6") failed: Property not found. retry in 1.224516689s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_396f2f68-ce7b-4018-8f2d-7aaea21e47d6
8882024-09-01T20:59:21.945ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b3dc62d6-da69-4341-a243-bce80bbf280e") failed: Property not found. retry in 596.462285ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b3dc62d6-da69-4341-a243-bce80bbf280e
8892024-09-01T20:59:22.338ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_94377199-0f3e-43d9-a00d-b82f88d48ad6") failed: Property not found. retry in 876.279759ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_94377199-0f3e-43d9-a00d-b82f88d48ad6
8902024-09-01T20:59:22.539ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_25121651-084b-4db4-8053-a44075741d18") failed: Property not found. retry in 741.651108ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_25121651-084b-4db4-8053-a44075741d18
8912024-09-01T20:59:22.550ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b3dc62d6-da69-4341-a243-bce80bbf280e") failed: Property not found. retry in 754.086051ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b3dc62d6-da69-4341-a243-bce80bbf280e
8922024-09-01T20:59:22.589ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0894f236-b210-4f67-b3d2-170d63897b72") failed: Property not found. retry in 796.926413ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0894f236-b210-4f67-b3d2-170d63897b72
8932024-09-01T20:59:23.170ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_396f2f68-ce7b-4018-8f2d-7aaea21e47d6") failed: Property not found. retry in 978.190059ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_396f2f68-ce7b-4018-8f2d-7aaea21e47d6
8942024-09-01T20:59:23.224ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_94377199-0f3e-43d9-a00d-b82f88d48ad6") failed: Property not found. retry in 1.35106348s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_94377199-0f3e-43d9-a00d-b82f88d48ad6
8952024-09-01T20:59:23.292ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_25121651-084b-4db4-8053-a44075741d18") failed: Property not found. retry in 630.494797ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_25121651-084b-4db4-8053-a44075741d18
8962024-09-01T20:59:23.312ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b3dc62d6-da69-4341-a243-bce80bbf280e") failed: Property not found. retry in 1.304325847s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b3dc62d6-da69-4341-a243-bce80bbf280e
8972024-09-01T20:59:23.395ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0894f236-b210-4f67-b3d2-170d63897b72") failed: Property not found. retry in 551.046766ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0894f236-b210-4f67-b3d2-170d63897b72
8982024-09-01T20:59:23.932ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_25121651-084b-4db4-8053-a44075741d18") failed: Property not found. retry in 1.332567929s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_25121651-084b-4db4-8053-a44075741d18
8992024-09-01T20:59:23.954ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0894f236-b210-4f67-b3d2-170d63897b72") failed: Property not found. retry in 612.644412ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0894f236-b210-4f67-b3d2-170d63897b72
9002024-09-01T20:59:24.159ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_396f2f68-ce7b-4018-8f2d-7aaea21e47d6") failed: Property not found. retry in 843.685096ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_396f2f68-ce7b-4018-8f2d-7aaea21e47d6
9012024-09-01T20:59:24.627ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b3dc62d6-da69-4341-a243-bce80bbf280e") failed: Property not found. retry in 986.938767ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b3dc62d6-da69-4341-a243-bce80bbf280e
9022024-09-01T20:59:25.636ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_cockroachdb_396f2f68-ce7b-4018-8f2d-7aaea21e47d6
9032024-09-01T20:59:25.636ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_cockroachdb_94377199-0f3e-43d9-a00d-b82f88d48ad6
9042024-09-01T20:59:25.636ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_cockroachdb_b3dc62d6-da69-4341-a243-bce80bbf280e
9052024-09-01T20:59:25.636ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_cockroachdb_25121651-084b-4db4-8053-a44075741d18
9062024-09-01T20:59:25.636ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_cockroachdb_0894f236-b210-4f67-b3d2-170d63897b72
9072024-09-01T20:59:25.636ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
9082024-09-01T20:59:25.636ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/c13a4209-582e-438f-b076-515144773dc5/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
9092024-09-01T20:59:25.637ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 16001768 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:43832 req_id = c0cdee7b-5c1a-4b34-bda3-54192621e7b4 response_code = 204 uri = /omicron-zones
9102024-09-01T20:59:25.670ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:52030
9112024-09-01T20:59:25.700ZINFOSledAgent (ServiceManager): Initializing CRDB Cluster - sending request to [fd00:1122:3344:101::5]:32221 file = sled-agent/src/services.rs:3686
9122024-09-01T20:59:35.887ZINFOSledAgent (ServiceManager): Formatting CRDB file = sled-agent/src/services.rs:3704
9132024-09-01T20:59:51.571ZINFOSledAgent (ServiceManager): Formatting CRDB - Completed file = sled-agent/src/services.rs:3727
9142024-09-01T20:59:51.571ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 25900066 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:52030 req_id = fb52b275-aa18-438f-b602-198fae7da369 response_code = 204 uri = /cockroachdb
9152024-09-01T20:59:51.572ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:55809
9162024-09-01T20:59:51.614ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(5), zones: [OmicronZoneConfig { id: 4410131d-d543-4f5e-b042-2b8f7df286aa, underlay_address: fd00:1122:3344:1::1, filesystem_pool: Some(ZpoolName { id: 067f6720-0053-47f5-be0b-12fc815674a2 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 067f6720-0053-47f5-be0b-12fc815674a2 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: c56694eb-8314-46fd-8d68-a8cdc82c2423, underlay_address: fd00:1122:3344:2::1, filesystem_pool: Some(ZpoolName { id: c65e07fa-2bd7-44d6-b309-46d959ab18fb (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: c65e07fa-2bd7-44d6-b309-46d959ab18fb (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 871d4f62-0910-41a9-b1f8-74b5d8646a65, underlay_address: fd00:1122:3344:3::1, filesystem_pool: Some(ZpoolName { id: e8cbb923-594d-495a-af32-03297bdb9d06 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: e8cbb923-594d-495a-af32-03297bdb9d06 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 85ac1227-6a8d-4cdc-a30a-2fc5ece363f8, underlay_address: fd00:1122:3344:101::1b, filesystem_pool: Some(ZpoolName { id: 6421be57-8f17-44b8-8266-2192ac87bae0 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 152ae1bb-1dfe-42d6-a186-30e49b450289, kind: Service { id: 85ac1227-6a8d-4cdc-a30a-2fc5ece363f8 }, name: Name("ntp-85ac1227-6a8d-4cdc-a30a-2fc5ece363f8"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 189, 209])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: b3dc62d6-da69-4341-a243-bce80bbf280e, underlay_address: fd00:1122:3344:101::3, filesystem_pool: Some(ZpoolName { id: 067f6720-0053-47f5-be0b-12fc815674a2 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 067f6720-0053-47f5-be0b-12fc815674a2 (zpool), kind: External } } } }, OmicronZoneConfig { id: 94377199-0f3e-43d9-a00d-b82f88d48ad6, underlay_address: fd00:1122:3344:101::4, filesystem_pool: Some(ZpoolName { id: c65e07fa-2bd7-44d6-b309-46d959ab18fb (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: c65e07fa-2bd7-44d6-b309-46d959ab18fb (zpool), kind: External } } } }, OmicronZoneConfig { id: 0894f236-b210-4f67-b3d2-170d63897b72, underlay_address: fd00:1122:3344:101::5, filesystem_pool: Some(ZpoolName { id: e8cbb923-594d-495a-af32-03297bdb9d06 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: e8cbb923-594d-495a-af32-03297bdb9d06 (zpool), kind: External } } } }, OmicronZoneConfig { id: 25121651-084b-4db4-8053-a44075741d18, underlay_address: fd00:1122:3344:101::6, filesystem_pool: Some(ZpoolName { id: b7b09d01-4b5a-46f4-a346-2e7b6a0e3462 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: b7b09d01-4b5a-46f4-a346-2e7b6a0e3462 (zpool), kind: External } } } }, OmicronZoneConfig { id: 396f2f68-ce7b-4018-8f2d-7aaea21e47d6, underlay_address: fd00:1122:3344:101::7, filesystem_pool: Some(ZpoolName { id: 14681c40-9950-4e30-8f61-ec7a2ca60e1c (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 14681c40-9950-4e30-8f61-ec7a2ca60e1c (zpool), kind: External } } } }, OmicronZoneConfig { id: d0653118-7c84-43ff-9e70-8fe4cb6a2a64, underlay_address: fd00:1122:3344:101::8, filesystem_pool: Some(ZpoolName { id: 067f6720-0053-47f5-be0b-12fc815674a2 (zpool), kind: External }), zone_type: ExternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 067f6720-0053-47f5-be0b-12fc815674a2 (zpool), kind: External } }, http_address: [fd00:1122:3344:101::8]:5353, dns_address: 10.151.2.164:53, nic: NetworkInterface { id: d1fabfad-8660-4a68-bb4b-5a632f9c053c, kind: Service { id: d0653118-7c84-43ff-9e70-8fe4cb6a2a64 }, name: Name("external-dns-d0653118-7c84-43ff-9e70-8fe4cb6a2a64"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 250, 139])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } } }, OmicronZoneConfig { id: e1c0d41c-ad07-4854-9798-d69fb32da3c7, underlay_address: fd00:1122:3344:101::9, filesystem_pool: Some(ZpoolName { id: c65e07fa-2bd7-44d6-b309-46d959ab18fb (zpool), kind: External }), zone_type: ExternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: c65e07fa-2bd7-44d6-b309-46d959ab18fb (zpool), kind: External } }, http_address: [fd00:1122:3344:101::9]:5353, dns_address: 10.151.2.165:53, nic: NetworkInterface { id: 30b8ab41-7fd4-46c2-a3ab-d5188430f837, kind: Service { id: e1c0d41c-ad07-4854-9798-d69fb32da3c7 }, name: Name("external-dns-e1c0d41c-ad07-4854-9798-d69fb32da3c7"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 141, 53])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } } }, OmicronZoneConfig { id: e6ddac78-e9f2-47ae-8121-95ee98f70ed8, underlay_address: fd00:1122:3344:101::a, filesystem_pool: Some(ZpoolName { id: 64079151-57af-4617-9647-1f77fe70ee41 (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::a]:12221, external_ip: 10.151.2.166, nic: NetworkInterface { id: 53c07f4a-42c7-4c20-a686-54474f5c9e46, kind: Service { id: e6ddac78-e9f2-47ae-8121-95ee98f70ed8 }, name: Name("nexus-e6ddac78-e9f2-47ae-8121-95ee98f70ed8"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 144, 191])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: fcd4de7a-43df-4ccf-a747-0c745f76751c, underlay_address: fd00:1122:3344:101::b, filesystem_pool: Some(ZpoolName { id: 067f6720-0053-47f5-be0b-12fc815674a2 (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::b]:12221, external_ip: 10.151.2.167, nic: NetworkInterface { id: d3ef2e45-8e9b-444a-a0ae-d65db7d825bb, kind: Service { id: fcd4de7a-43df-4ccf-a747-0c745f76751c }, name: Name("nexus-fcd4de7a-43df-4ccf-a747-0c745f76751c"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 204, 184])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: f9b91117-5154-4cc0-9de9-2e9efaaad0e6, underlay_address: fd00:1122:3344:101::c, filesystem_pool: Some(ZpoolName { id: 8d3c930e-5dba-4325-82af-75dd024bc8cc (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::c]:12221, external_ip: 10.151.2.168, nic: NetworkInterface { id: bae4dcf4-d7eb-4337-81e0-130002ad9304, kind: Service { id: f9b91117-5154-4cc0-9de9-2e9efaaad0e6 }, name: Name("nexus-f9b91117-5154-4cc0-9de9-2e9efaaad0e6"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 194, 40])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: 7d3d53d3-b0f8-4024-a319-1703ede11193, underlay_address: fd00:1122:3344:101::d, filesystem_pool: Some(ZpoolName { id: 14681c40-9950-4e30-8f61-ec7a2ca60e1c (zpool), kind: External }), zone_type: Oximeter { address: [fd00:1122:3344:101::d]:12223 } }, OmicronZoneConfig { id: 03bcf173-e5c2-4545-ac63-b6f41f5cbdda, underlay_address: fd00:1122:3344:101::e, filesystem_pool: Some(ZpoolName { id: 067f6720-0053-47f5-be0b-12fc815674a2 (zpool), kind: External }), zone_type: Clickhouse { address: [fd00:1122:3344:101::e]:8123, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 067f6720-0053-47f5-be0b-12fc815674a2 (zpool), kind: External } } } }, OmicronZoneConfig { id: 95103d07-7ccb-4d09-b928-56c0fb8e4bb3, underlay_address: fd00:1122:3344:101::f, filesystem_pool: Some(ZpoolName { id: 8d3c930e-5dba-4325-82af-75dd024bc8cc (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::f]:17000 } }, OmicronZoneConfig { id: 34c7db96-4f54-435b-bea8-b34ac92b151b, underlay_address: fd00:1122:3344:101::10, filesystem_pool: Some(ZpoolName { id: 8d3c930e-5dba-4325-82af-75dd024bc8cc (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::10]:17000 } }, OmicronZoneConfig { id: e7ead4f1-17b4-4c7d-843c-7f2e488501a0, underlay_address: fd00:1122:3344:101::11, filesystem_pool: Some(ZpoolName { id: 7a19116b-f2a7-4643-8d6b-df9db59b345a (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::11]:17000 } }, OmicronZoneConfig { id: 0da3314b-38b4-4938-8831-8ec6d7490158, underlay_address: fd00:1122:3344:101::12, filesystem_pool: Some(ZpoolName { id: 067f6720-0053-47f5-be0b-12fc815674a2 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::12]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 067f6720-0053-47f5-be0b-12fc815674a2 (zpool), kind: External } } } }, OmicronZoneConfig { id: 4091dac0-a803-4f41-a822-732fdb74b5db, underlay_address: fd00:1122:3344:101::13, filesystem_pool: Some(ZpoolName { id: c65e07fa-2bd7-44d6-b309-46d959ab18fb (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::13]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: c65e07fa-2bd7-44d6-b309-46d959ab18fb (zpool), kind: External } } } }, OmicronZoneConfig { id: a7186ed2-a4e0-4db3-9300-6b0a04c483af, underlay_address: fd00:1122:3344:101::14, filesystem_pool: Some(ZpoolName { id: e8cbb923-594d-495a-af32-03297bdb9d06 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::14]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: e8cbb923-594d-495a-af32-03297bdb9d06 (zpool), kind: External } } } }, OmicronZoneConfig { id: cf7f9d18-6fc4-4ee0-b35d-57ed8769bf3d, underlay_address: fd00:1122:3344:101::15, filesystem_pool: Some(ZpoolName { id: b7b09d01-4b5a-46f4-a346-2e7b6a0e3462 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::15]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: b7b09d01-4b5a-46f4-a346-2e7b6a0e3462 (zpool), kind: External } } } }, OmicronZoneConfig { id: a0ecfdff-5e02-4b4f-863f-6f55a5749ea0, underlay_address: fd00:1122:3344:101::16, filesystem_pool: Some(ZpoolName { id: 14681c40-9950-4e30-8f61-ec7a2ca60e1c (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::16]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 14681c40-9950-4e30-8f61-ec7a2ca60e1c (zpool), kind: External } } } }, OmicronZoneConfig { id: b4673d99-2ce4-485e-a28e-9c3588338668, underlay_address: fd00:1122:3344:101::17, filesystem_pool: Some(ZpoolName { id: 8d3c930e-5dba-4325-82af-75dd024bc8cc (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::17]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8d3c930e-5dba-4325-82af-75dd024bc8cc (zpool), kind: External } } } }, OmicronZoneConfig { id: 6fe8da63-5815-4e50-aa5f-56ee726aff73, underlay_address: fd00:1122:3344:101::18, filesystem_pool: Some(ZpoolName { id: 6421be57-8f17-44b8-8266-2192ac87bae0 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::18]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 6421be57-8f17-44b8-8266-2192ac87bae0 (zpool), kind: External } } } }, OmicronZoneConfig { id: 93d2e0a4-500d-430c-b7ae-5ba369a25e48, underlay_address: fd00:1122:3344:101::19, filesystem_pool: Some(ZpoolName { id: 64079151-57af-4617-9647-1f77fe70ee41 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::19]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 64079151-57af-4617-9647-1f77fe70ee41 (zpool), kind: External } } } }, OmicronZoneConfig { id: 854f46a4-dab9-4716-b39d-9f437db37063, underlay_address: fd00:1122:3344:101::1a, filesystem_pool: Some(ZpoolName { id: 7a19116b-f2a7-4643-8d6b-df9db59b345a (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::1a]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 7a19116b-f2a7-4643-8d6b-df9db59b345a (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:438 sled_address = [fd00:1122:3344:101::1]:12345
9172024-09-01T20:59:51.614ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:38923
9182024-09-01T20:59:51.615ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 4410131d-d543-4f5e-b042-2b8f7df286aa, dataset_name: DatasetName { pool_name: ZpoolName { id: 067f6720-0053-47f5-be0b-12fc815674a2 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
9192024-09-01T20:59:51.615ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 4410131d-d543-4f5e-b042-2b8f7df286aa, dataset_name: DatasetName { pool_name: ZpoolName { id: 067f6720-0053-47f5-be0b-12fc815674a2 (zpool), kind: External }, kind: InternalDns }, responder: ... }
9202024-09-01T20:59:51.640ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c56694eb-8314-46fd-8d68-a8cdc82c2423, dataset_name: DatasetName { pool_name: ZpoolName { id: c65e07fa-2bd7-44d6-b309-46d959ab18fb (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
9212024-09-01T20:59:51.640ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: c56694eb-8314-46fd-8d68-a8cdc82c2423, dataset_name: DatasetName { pool_name: ZpoolName { id: c65e07fa-2bd7-44d6-b309-46d959ab18fb (zpool), kind: External }, kind: InternalDns }, responder: ... }
9222024-09-01T20:59:51.661ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 871d4f62-0910-41a9-b1f8-74b5d8646a65, dataset_name: DatasetName { pool_name: ZpoolName { id: e8cbb923-594d-495a-af32-03297bdb9d06 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
9232024-09-01T20:59:51.661ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 871d4f62-0910-41a9-b1f8-74b5d8646a65, dataset_name: DatasetName { pool_name: ZpoolName { id: e8cbb923-594d-495a-af32-03297bdb9d06 (zpool), kind: External }, kind: InternalDns }, responder: ... }
9242024-09-01T20:59:51.681ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: b3dc62d6-da69-4341-a243-bce80bbf280e, dataset_name: DatasetName { pool_name: ZpoolName { id: 067f6720-0053-47f5-be0b-12fc815674a2 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
9252024-09-01T20:59:51.681ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: b3dc62d6-da69-4341-a243-bce80bbf280e, dataset_name: DatasetName { pool_name: ZpoolName { id: 067f6720-0053-47f5-be0b-12fc815674a2 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9262024-09-01T20:59:51.702ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 94377199-0f3e-43d9-a00d-b82f88d48ad6, dataset_name: DatasetName { pool_name: ZpoolName { id: c65e07fa-2bd7-44d6-b309-46d959ab18fb (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
9272024-09-01T20:59:51.702ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 94377199-0f3e-43d9-a00d-b82f88d48ad6, dataset_name: DatasetName { pool_name: ZpoolName { id: c65e07fa-2bd7-44d6-b309-46d959ab18fb (zpool), kind: External }, kind: Cockroach }, responder: ... }
9282024-09-01T20:59:51.722ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 0894f236-b210-4f67-b3d2-170d63897b72, dataset_name: DatasetName { pool_name: ZpoolName { id: e8cbb923-594d-495a-af32-03297bdb9d06 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
9292024-09-01T20:59:51.722ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 0894f236-b210-4f67-b3d2-170d63897b72, dataset_name: DatasetName { pool_name: ZpoolName { id: e8cbb923-594d-495a-af32-03297bdb9d06 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9302024-09-01T20:59:51.742ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 25121651-084b-4db4-8053-a44075741d18, dataset_name: DatasetName { pool_name: ZpoolName { id: b7b09d01-4b5a-46f4-a346-2e7b6a0e3462 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
9312024-09-01T20:59:51.742ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 25121651-084b-4db4-8053-a44075741d18, dataset_name: DatasetName { pool_name: ZpoolName { id: b7b09d01-4b5a-46f4-a346-2e7b6a0e3462 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9322024-09-01T20:59:51.762ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 396f2f68-ce7b-4018-8f2d-7aaea21e47d6, dataset_name: DatasetName { pool_name: ZpoolName { id: 14681c40-9950-4e30-8f61-ec7a2ca60e1c (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
9332024-09-01T20:59:51.762ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 396f2f68-ce7b-4018-8f2d-7aaea21e47d6, dataset_name: DatasetName { pool_name: ZpoolName { id: 14681c40-9950-4e30-8f61-ec7a2ca60e1c (zpool), kind: External }, kind: Cockroach }, responder: ... }
9342024-09-01T20:59:51.780ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: d0653118-7c84-43ff-9e70-8fe4cb6a2a64, dataset_name: DatasetName { pool_name: ZpoolName { id: 067f6720-0053-47f5-be0b-12fc815674a2 (zpool), kind: External }, kind: ExternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
9352024-09-01T20:59:51.780ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: d0653118-7c84-43ff-9e70-8fe4cb6a2a64, dataset_name: DatasetName { pool_name: ZpoolName { id: 067f6720-0053-47f5-be0b-12fc815674a2 (zpool), kind: External }, kind: ExternalDns }, responder: ... }
9362024-09-01T20:59:51.933ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: e1c0d41c-ad07-4854-9798-d69fb32da3c7, dataset_name: DatasetName { pool_name: ZpoolName { id: c65e07fa-2bd7-44d6-b309-46d959ab18fb (zpool), kind: External }, kind: ExternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
9372024-09-01T20:59:51.933ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: e1c0d41c-ad07-4854-9798-d69fb32da3c7, dataset_name: DatasetName { pool_name: ZpoolName { id: c65e07fa-2bd7-44d6-b309-46d959ab18fb (zpool), kind: External }, kind: ExternalDns }, responder: ... }
9382024-09-01T20:59:52.158ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 03bcf173-e5c2-4545-ac63-b6f41f5cbdda, dataset_name: DatasetName { pool_name: ZpoolName { id: 067f6720-0053-47f5-be0b-12fc815674a2 (zpool), kind: External }, kind: Clickhouse }, responder: ... }) file = sled-storage/src/manager.rs:451
9392024-09-01T20:59:52.158ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 03bcf173-e5c2-4545-ac63-b6f41f5cbdda, dataset_name: DatasetName { pool_name: ZpoolName { id: 067f6720-0053-47f5-be0b-12fc815674a2 (zpool), kind: External }, kind: Clickhouse }, responder: ... }
9402024-09-01T20:59:52.269ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 0da3314b-38b4-4938-8831-8ec6d7490158, dataset_name: DatasetName { pool_name: ZpoolName { id: 067f6720-0053-47f5-be0b-12fc815674a2 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:451
9412024-09-01T20:59:52.269ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 0da3314b-38b4-4938-8831-8ec6d7490158, dataset_name: DatasetName { pool_name: ZpoolName { id: 067f6720-0053-47f5-be0b-12fc815674a2 (zpool), kind: External }, kind: Crucible }, responder: ... }
9422024-09-01T20:59:52.419ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 4091dac0-a803-4f41-a822-732fdb74b5db, dataset_name: DatasetName { pool_name: ZpoolName { id: c65e07fa-2bd7-44d6-b309-46d959ab18fb (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:451
9432024-09-01T20:59:52.419ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 4091dac0-a803-4f41-a822-732fdb74b5db, dataset_name: DatasetName { pool_name: ZpoolName { id: c65e07fa-2bd7-44d6-b309-46d959ab18fb (zpool), kind: External }, kind: Crucible }, responder: ... }
9442024-09-01T20:59:52.560ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: a7186ed2-a4e0-4db3-9300-6b0a04c483af, dataset_name: DatasetName { pool_name: ZpoolName { id: e8cbb923-594d-495a-af32-03297bdb9d06 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:451
9452024-09-01T20:59:52.561ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: a7186ed2-a4e0-4db3-9300-6b0a04c483af, dataset_name: DatasetName { pool_name: ZpoolName { id: e8cbb923-594d-495a-af32-03297bdb9d06 (zpool), kind: External }, kind: Crucible }, responder: ... }
9462024-09-01T20:59:52.716ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: cf7f9d18-6fc4-4ee0-b35d-57ed8769bf3d, dataset_name: DatasetName { pool_name: ZpoolName { id: b7b09d01-4b5a-46f4-a346-2e7b6a0e3462 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:451
9472024-09-01T20:59:52.716ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: cf7f9d18-6fc4-4ee0-b35d-57ed8769bf3d, dataset_name: DatasetName { pool_name: ZpoolName { id: b7b09d01-4b5a-46f4-a346-2e7b6a0e3462 (zpool), kind: External }, kind: Crucible }, responder: ... }
9482024-09-01T20:59:52.806ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: a0ecfdff-5e02-4b4f-863f-6f55a5749ea0, dataset_name: DatasetName { pool_name: ZpoolName { id: 14681c40-9950-4e30-8f61-ec7a2ca60e1c (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:451
9492024-09-01T20:59:52.806ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: a0ecfdff-5e02-4b4f-863f-6f55a5749ea0, dataset_name: DatasetName { pool_name: ZpoolName { id: 14681c40-9950-4e30-8f61-ec7a2ca60e1c (zpool), kind: External }, kind: Crucible }, responder: ... }
9502024-09-01T20:59:52.921ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: b4673d99-2ce4-485e-a28e-9c3588338668, dataset_name: DatasetName { pool_name: ZpoolName { id: 8d3c930e-5dba-4325-82af-75dd024bc8cc (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:451
9512024-09-01T20:59:52.921ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: b4673d99-2ce4-485e-a28e-9c3588338668, dataset_name: DatasetName { pool_name: ZpoolName { id: 8d3c930e-5dba-4325-82af-75dd024bc8cc (zpool), kind: External }, kind: Crucible }, responder: ... }
9522024-09-01T20:59:53.080ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 6fe8da63-5815-4e50-aa5f-56ee726aff73, dataset_name: DatasetName { pool_name: ZpoolName { id: 6421be57-8f17-44b8-8266-2192ac87bae0 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:451
9532024-09-01T20:59:53.080ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 6fe8da63-5815-4e50-aa5f-56ee726aff73, dataset_name: DatasetName { pool_name: ZpoolName { id: 6421be57-8f17-44b8-8266-2192ac87bae0 (zpool), kind: External }, kind: Crucible }, responder: ... }
9542024-09-01T20:59:53.180ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 93d2e0a4-500d-430c-b7ae-5ba369a25e48, dataset_name: DatasetName { pool_name: ZpoolName { id: 64079151-57af-4617-9647-1f77fe70ee41 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:451
9552024-09-01T20:59:53.180ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 93d2e0a4-500d-430c-b7ae-5ba369a25e48, dataset_name: DatasetName { pool_name: ZpoolName { id: 64079151-57af-4617-9647-1f77fe70ee41 (zpool), kind: External }, kind: Crucible }, responder: ... }
9562024-09-01T20:59:53.320ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 854f46a4-dab9-4716-b39d-9f437db37063, dataset_name: DatasetName { pool_name: ZpoolName { id: 7a19116b-f2a7-4643-8d6b-df9db59b345a (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:451
9572024-09-01T20:59:53.320ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 854f46a4-dab9-4716-b39d-9f437db37063, dataset_name: DatasetName { pool_name: ZpoolName { id: 7a19116b-f2a7-4643-8d6b-df9db59b345a (zpool), kind: External }, kind: Crucible }, responder: ... }
9582024-09-01T20:59:53.462ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
9592024-09-01T20:59:53.462ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e/config/omicron-zones.json file = common/src/ledger.rs:177
9602024-09-01T20:59:53.462ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/c13a4209-582e-438f-b076-515144773dc5/config/omicron-zones.json file = common/src/ledger.rs:177
9612024-09-01T20:59:53.462ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
9622024-09-01T20:59:53.574ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
9632024-09-01T20:59:53.574ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
9642024-09-01T20:59:53.574ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
9652024-09-01T20:59:53.574ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
9662024-09-01T20:59:53.575ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
9672024-09-01T20:59:53.588ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
9682024-09-01T20:59:53.588ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
9692024-09-01T20:59:53.612ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
9702024-09-01T20:59:53.612ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
9712024-09-01T20:59:53.624ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
9722024-09-01T20:59:53.761ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
9732024-09-01T20:59:53.761ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
9742024-09-01T20:59:53.761ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
9752024-09-01T20:59:53.821ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
9762024-09-01T20:59:53.821ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
9772024-09-01T20:59:53.821ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
9782024-09-01T20:59:53.821ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
9792024-09-01T20:59:53.821ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
9802024-09-01T20:59:53.821ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
9812024-09-01T20:59:53.821ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_oximeter_7d3d53d3-b0f8-4024-a319-1703ede11193 file = illumos-utils/src/zone.rs:316
9822024-09-01T20:59:53.827ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_95103d07-7ccb-4d09-b928-56c0fb8e4bb3 file = illumos-utils/src/zone.rs:316
9832024-09-01T20:59:53.832ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_a7186ed2-a4e0-4db3-9300-6b0a04c483af file = illumos-utils/src/zone.rs:316
9842024-09-01T20:59:53.837ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_34c7db96-4f54-435b-bea8-b34ac92b151b file = illumos-utils/src/zone.rs:316
9852024-09-01T20:59:53.842ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_4091dac0-a803-4f41-a822-732fdb74b5db file = illumos-utils/src/zone.rs:316
9862024-09-01T20:59:53.847ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_cf7f9d18-6fc4-4ee0-b35d-57ed8769bf3d file = illumos-utils/src/zone.rs:316
9872024-09-01T20:59:54.007ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_oximeter_7d3d53d3-b0f8-4024-a319-1703ede11193 file = illumos-utils/src/zone.rs:354
9882024-09-01T20:59:54.013ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_95103d07-7ccb-4d09-b928-56c0fb8e4bb3 file = illumos-utils/src/zone.rs:354
9892024-09-01T20:59:54.018ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_a7186ed2-a4e0-4db3-9300-6b0a04c483af file = illumos-utils/src/zone.rs:354
9902024-09-01T20:59:54.023ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_34c7db96-4f54-435b-bea8-b34ac92b151b file = illumos-utils/src/zone.rs:354
9912024-09-01T20:59:54.027ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_4091dac0-a803-4f41-a822-732fdb74b5db file = illumos-utils/src/zone.rs:354
9922024-09-01T20:59:54.032ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_cf7f9d18-6fc4-4ee0-b35d-57ed8769bf3d file = illumos-utils/src/zone.rs:354
9932024-09-01T20:59:54.037ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_a0ecfdff-5e02-4b4f-863f-6f55a5749ea0 file = illumos-utils/src/zone.rs:316
9942024-09-01T20:59:54.043ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_854f46a4-dab9-4716-b39d-9f437db37063 file = illumos-utils/src/zone.rs:316
9952024-09-01T20:59:54.048ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_clickhouse_03bcf173-e5c2-4545-ac63-b6f41f5cbdda file = illumos-utils/src/zone.rs:316
9962024-09-01T20:59:54.054ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_0da3314b-38b4-4938-8831-8ec6d7490158 file = illumos-utils/src/zone.rs:316
9972024-09-01T20:59:54.060ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_6fe8da63-5815-4e50-aa5f-56ee726aff73 file = illumos-utils/src/zone.rs:316
9982024-09-01T20:59:54.065ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_e7ead4f1-17b4-4c7d-843c-7f2e488501a0 file = illumos-utils/src/zone.rs:316
9992024-09-01T20:59:54.071ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_b4673d99-2ce4-485e-a28e-9c3588338668 file = illumos-utils/src/zone.rs:316
10002024-09-01T20:59:54.077ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_93d2e0a4-500d-430c-b7ae-5ba369a25e48 file = illumos-utils/src/zone.rs:316
10012024-09-01T20:59:54.118ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
10022024-09-01T20:59:54.151ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
10032024-09-01T20:59:54.151ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_a0ecfdff-5e02-4b4f-863f-6f55a5749ea0 file = illumos-utils/src/zone.rs:354
10042024-09-01T20:59:54.157ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_854f46a4-dab9-4716-b39d-9f437db37063 file = illumos-utils/src/zone.rs:354
10052024-09-01T20:59:54.163ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_clickhouse_03bcf173-e5c2-4545-ac63-b6f41f5cbdda file = illumos-utils/src/zone.rs:354
10062024-09-01T20:59:54.168ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_0da3314b-38b4-4938-8831-8ec6d7490158 file = illumos-utils/src/zone.rs:354
10072024-09-01T20:59:54.175ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_6fe8da63-5815-4e50-aa5f-56ee726aff73 file = illumos-utils/src/zone.rs:354
10082024-09-01T20:59:54.180ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_e7ead4f1-17b4-4c7d-843c-7f2e488501a0 file = illumos-utils/src/zone.rs:354
10092024-09-01T20:59:54.186ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_b4673d99-2ce4-485e-a28e-9c3588338668 file = illumos-utils/src/zone.rs:354
10102024-09-01T20:59:54.192ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_93d2e0a4-500d-430c-b7ae-5ba369a25e48 file = illumos-utils/src/zone.rs:354
10112024-09-01T20:59:54.236ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
10122024-09-01T20:59:54.277ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
10132024-09-01T20:59:54.279ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
10142024-09-01T20:59:54.279ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
10152024-09-01T20:59:54.310ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte1", ip: 172.30.2.5, mac: MacAddr6([168, 64, 37, 255, 144, 191]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
10162024-09-01T20:59:54.310ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1234 zone_type = nexus
10172024-09-01T20:59:54.312ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
10182024-09-01T20:59:54.312ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
10192024-09-01T20:59:54.343ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte2", ip: 172.30.1.5, mac: MacAddr6([168, 64, 37, 255, 250, 139]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.1.1 } }) }
10202024-09-01T20:59:54.343ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1234 zone_type = external_dns
10212024-09-01T20:59:54.375ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
10222024-09-01T20:59:54.376ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
10232024-09-01T20:59:54.376ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
10242024-09-01T20:59:54.417ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte3", ip: 172.30.2.7, mac: MacAddr6([168, 64, 37, 255, 194, 40]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
10252024-09-01T20:59:54.417ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1234 zone_type = nexus
10262024-09-01T20:59:54.420ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
10272024-09-01T20:59:54.420ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
10282024-09-01T20:59:54.475ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte4", ip: 172.30.1.6, mac: MacAddr6([168, 64, 37, 255, 141, 53]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.1.1 } }) }
10292024-09-01T20:59:54.475ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1234 zone_type = external_dns
10302024-09-01T20:59:54.475ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.164 file = clients/dpd-client/src/lib.rs:83
10312024-09-01T20:59:54.475ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.164 file = clients/dpd-client/src/lib.rs:90
10322024-09-01T20:59:54.477ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
10332024-09-01T20:59:54.477ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
10342024-09-01T20:59:54.530ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte5", ip: 172.30.2.6, mac: MacAddr6([168, 64, 37, 255, 204, 184]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
10352024-09-01T20:59:54.530ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1234 zone_type = nexus
10362024-09-01T20:59:54.530ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.166 file = clients/dpd-client/src/lib.rs:83
10372024-09-01T20:59:54.530ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.166 file = clients/dpd-client/src/lib.rs:90
10382024-09-01T20:59:54.530ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.164 file = clients/dpd-client/src/lib.rs:118
10392024-09-01T20:59:54.532ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
10402024-09-01T20:59:54.590ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.167 file = clients/dpd-client/src/lib.rs:83
10412024-09-01T20:59:54.590ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.167 file = clients/dpd-client/src/lib.rs:90
10422024-09-01T20:59:54.591ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.166 file = clients/dpd-client/src/lib.rs:118
10432024-09-01T20:59:54.596ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.168 file = clients/dpd-client/src/lib.rs:83
10442024-09-01T20:59:54.596ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.168 file = clients/dpd-client/src/lib.rs:90
10452024-09-01T20:59:54.596ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
10462024-09-01T20:59:54.598ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.165 file = clients/dpd-client/src/lib.rs:83
10472024-09-01T20:59:54.598ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.165 file = clients/dpd-client/src/lib.rs:90
10482024-09-01T20:59:54.598ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.167 file = clients/dpd-client/src/lib.rs:118
10492024-09-01T20:59:54.599ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
10502024-09-01T20:59:54.662ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.168 file = clients/dpd-client/src/lib.rs:118
10512024-09-01T20:59:54.666ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.165 file = clients/dpd-client/src/lib.rs:118
10522024-09-01T20:59:54.669ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
10532024-09-01T20:59:54.669ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
10542024-09-01T20:59:54.669ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_external_dns_d0653118-7c84-43ff-9e70-8fe4cb6a2a64 file = illumos-utils/src/zone.rs:316
10552024-09-01T20:59:54.726ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_e6ddac78-e9f2-47ae-8121-95ee98f70ed8 file = illumos-utils/src/zone.rs:316
10562024-09-01T20:59:54.817ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_external_dns_d0653118-7c84-43ff-9e70-8fe4cb6a2a64 file = illumos-utils/src/zone.rs:354
10572024-09-01T20:59:54.825ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_fcd4de7a-43df-4ccf-a747-0c745f76751c file = illumos-utils/src/zone.rs:316
10582024-09-01T20:59:54.834ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_e6ddac78-e9f2-47ae-8121-95ee98f70ed8 file = illumos-utils/src/zone.rs:354
10592024-09-01T20:59:54.846ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_f9b91117-5154-4cc0-9de9-2e9efaaad0e6 file = illumos-utils/src/zone.rs:316
10602024-09-01T20:59:54.858ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_external_dns_e1c0d41c-ad07-4854-9798-d69fb32da3c7 file = illumos-utils/src/zone.rs:316
10612024-09-01T20:59:54.867ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_fcd4de7a-43df-4ccf-a747-0c745f76751c file = illumos-utils/src/zone.rs:354
10622024-09-01T20:59:54.884ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_f9b91117-5154-4cc0-9de9-2e9efaaad0e6 file = illumos-utils/src/zone.rs:354
10632024-09-01T20:59:54.896ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_external_dns_e1c0d41c-ad07-4854-9798-d69fb32da3c7 file = illumos-utils/src/zone.rs:354
10642024-09-01T20:59:55.654ZINFOSledAgent (ServiceManager): Profile for oxz_oximeter_7d3d53d3-b0f8-4024-a319-1703ede11193: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService10'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::d'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/oximeter"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="id" value='7d3d53d3-b0f8-4024-a319-1703ede11193'/> <propval type="astring" name="address" value='[fd00:1122:3344:101::d]:12223'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10652024-09-01T20:59:55.654ZINFOSledAgent (ServiceManager): Booting oxz_oximeter_7d3d53d3-b0f8-4024-a319-1703ede11193 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_oximeter_7d3d53d3-b0f8-4024-a319-1703ede11193
10662024-09-01T20:59:56.096ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_34c7db96-4f54-435b-bea8-b34ac92b151b: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService13'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::10'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::10'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10672024-09-01T20:59:56.096ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_34c7db96-4f54-435b-bea8-b34ac92b151b zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_34c7db96-4f54-435b-bea8-b34ac92b151b
10682024-09-01T20:59:56.361ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_95103d07-7ccb-4d09-b928-56c0fb8e4bb3: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService11'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::f'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::f'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10692024-09-01T20:59:56.362ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_95103d07-7ccb-4d09-b928-56c0fb8e4bb3 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_95103d07-7ccb-4d09-b928-56c0fb8e4bb3
10702024-09-01T20:59:56.377ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_a7186ed2-a4e0-4db3-9300-6b0a04c483af: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService12'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::14'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_e8cbb923-594d-495a-af32-03297bdb9d06/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::14'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='42702d4e-3732-4da4-8263-1fff6e773380'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10712024-09-01T20:59:56.378ZINFOSledAgent (ServiceManager): Booting oxz_crucible_a7186ed2-a4e0-4db3-9300-6b0a04c483af zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_a7186ed2-a4e0-4db3-9300-6b0a04c483af
10722024-09-01T20:59:56.393ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_4091dac0-a803-4f41-a822-732fdb74b5db: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService14'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::13'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_c65e07fa-2bd7-44d6-b309-46d959ab18fb/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::13'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='065dd0b5-0dc9-42e1-81a4-c0161560b95c'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10732024-09-01T20:59:56.399ZINFOSledAgent (ServiceManager): Booting oxz_crucible_4091dac0-a803-4f41-a822-732fdb74b5db zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_4091dac0-a803-4f41-a822-732fdb74b5db
10742024-09-01T20:59:56.608ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_cf7f9d18-6fc4-4ee0-b35d-57ed8769bf3d: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService15'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::15'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_b7b09d01-4b5a-46f4-a346-2e7b6a0e3462/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::15'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='d54a40bc-fadb-4053-ab40-c6cf5518b83e'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10752024-09-01T20:59:56.608ZINFOSledAgent (ServiceManager): Booting oxz_crucible_cf7f9d18-6fc4-4ee0-b35d-57ed8769bf3d zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_cf7f9d18-6fc4-4ee0-b35d-57ed8769bf3d
10762024-09-01T20:59:56.675ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_7d3d53d3-b0f8-4024-a319-1703ede11193") failed: Property not found. retry in 71.131773ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_7d3d53d3-b0f8-4024-a319-1703ede11193
10772024-09-01T20:59:56.695ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_93d2e0a4-500d-430c-b7ae-5ba369a25e48: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService23'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::19'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_64079151-57af-4617-9647-1f77fe70ee41/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::19'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='da3c1ba1-33ef-4564-b373-dd9de4c264fd'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10782024-09-01T20:59:56.695ZINFOSledAgent (ServiceManager): Booting oxz_crucible_93d2e0a4-500d-430c-b7ae-5ba369a25e48 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_93d2e0a4-500d-430c-b7ae-5ba369a25e48
10792024-09-01T20:59:56.794ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_7d3d53d3-b0f8-4024-a319-1703ede11193") failed: Property not found. retry in 105.330034ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_7d3d53d3-b0f8-4024-a319-1703ede11193
10802024-09-01T20:59:56.836ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_95103d07-7ccb-4d09-b928-56c0fb8e4bb3") failed: Property not found. retry in 54.014729ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_95103d07-7ccb-4d09-b928-56c0fb8e4bb3
10812024-09-01T20:59:56.892ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_34c7db96-4f54-435b-bea8-b34ac92b151b") failed: Property not found. retry in 72.814383ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_34c7db96-4f54-435b-bea8-b34ac92b151b
10822024-09-01T20:59:56.893ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_6fe8da63-5815-4e50-aa5f-56ee726aff73: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService20'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::18'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_6421be57-8f17-44b8-8266-2192ac87bae0/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::18'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='f6866a06-eea4-4f6c-9e81-ff3de5e88c8b'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10832024-09-01T20:59:56.937ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_95103d07-7ccb-4d09-b928-56c0fb8e4bb3") failed: Property not found. retry in 120.574942ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_95103d07-7ccb-4d09-b928-56c0fb8e4bb3
10842024-09-01T20:59:56.937ZINFOSledAgent (ServiceManager): Booting oxz_crucible_6fe8da63-5815-4e50-aa5f-56ee726aff73 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_6fe8da63-5815-4e50-aa5f-56ee726aff73
10852024-09-01T20:59:56.970ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_7d3d53d3-b0f8-4024-a319-1703ede11193") failed: Property not found. retry in 257.75646ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_7d3d53d3-b0f8-4024-a319-1703ede11193
10862024-09-01T20:59:56.980ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a7186ed2-a4e0-4db3-9300-6b0a04c483af") failed: Property not found. retry in 63.298503ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a7186ed2-a4e0-4db3-9300-6b0a04c483af
10872024-09-01T20:59:56.992ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_34c7db96-4f54-435b-bea8-b34ac92b151b") failed: Property not found. retry in 132.116414ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_34c7db96-4f54-435b-bea8-b34ac92b151b
10882024-09-01T20:59:57.004ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4091dac0-a803-4f41-a822-732fdb74b5db") failed: Property not found. retry in 45.40735ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4091dac0-a803-4f41-a822-732fdb74b5db
10892024-09-01T20:59:57.057ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a7186ed2-a4e0-4db3-9300-6b0a04c483af") failed: Property not found. retry in 100.02902ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a7186ed2-a4e0-4db3-9300-6b0a04c483af
10902024-09-01T20:59:57.068ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4091dac0-a803-4f41-a822-732fdb74b5db") failed: Property not found. retry in 98.287585ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4091dac0-a803-4f41-a822-732fdb74b5db
10912024-09-01T20:59:57.081ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_95103d07-7ccb-4d09-b928-56c0fb8e4bb3") failed: Property not found. retry in 221.611072ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_95103d07-7ccb-4d09-b928-56c0fb8e4bb3
10922024-09-01T20:59:57.095ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cf7f9d18-6fc4-4ee0-b35d-57ed8769bf3d") failed: Property not found. retry in 33.042769ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cf7f9d18-6fc4-4ee0-b35d-57ed8769bf3d
10932024-09-01T20:59:57.114ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_93d2e0a4-500d-430c-b7ae-5ba369a25e48") failed: Property not found. retry in 67.855535ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_93d2e0a4-500d-430c-b7ae-5ba369a25e48
10942024-09-01T20:59:57.147ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_34c7db96-4f54-435b-bea8-b34ac92b151b") failed: Property not found. retry in 199.567353ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_34c7db96-4f54-435b-bea8-b34ac92b151b
10952024-09-01T20:59:57.160ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cf7f9d18-6fc4-4ee0-b35d-57ed8769bf3d") failed: Property not found. retry in 122.894359ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cf7f9d18-6fc4-4ee0-b35d-57ed8769bf3d
10962024-09-01T20:59:57.177ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a7186ed2-a4e0-4db3-9300-6b0a04c483af") failed: Property not found. retry in 282.768755ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a7186ed2-a4e0-4db3-9300-6b0a04c483af
10972024-09-01T20:59:57.192ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4091dac0-a803-4f41-a822-732fdb74b5db") failed: Property not found. retry in 244.284483ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4091dac0-a803-4f41-a822-732fdb74b5db
10982024-09-01T20:59:57.202ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6fe8da63-5815-4e50-aa5f-56ee726aff73") failed: Property not found. retry in 59.258778ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6fe8da63-5815-4e50-aa5f-56ee726aff73
10992024-09-01T20:59:57.212ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_93d2e0a4-500d-430c-b7ae-5ba369a25e48") failed: Property not found. retry in 80.327584ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_93d2e0a4-500d-430c-b7ae-5ba369a25e48
11002024-09-01T20:59:57.242ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_7d3d53d3-b0f8-4024-a319-1703ede11193") failed: Property not found. retry in 340.168898ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_7d3d53d3-b0f8-4024-a319-1703ede11193
11012024-09-01T21:00:03.502ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6fe8da63-5815-4e50-aa5f-56ee726aff73") failed: Property not found. retry in 102.581686ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6fe8da63-5815-4e50-aa5f-56ee726aff73
11022024-09-01T21:00:03.811ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_7d3d53d3-b0f8-4024-a319-1703ede11193") failed: Property not found. retry in 1.051731958s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_7d3d53d3-b0f8-4024-a319-1703ede11193
11032024-09-01T21:00:04.033ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_95103d07-7ccb-4d09-b928-56c0fb8e4bb3") failed: Property not found. retry in 416.281013ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_95103d07-7ccb-4d09-b928-56c0fb8e4bb3
11042024-09-01T21:00:04.237ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a7186ed2-a4e0-4db3-9300-6b0a04c483af") failed: Property not found. retry in 460.059129ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a7186ed2-a4e0-4db3-9300-6b0a04c483af
11052024-09-01T21:00:04.285ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_34c7db96-4f54-435b-bea8-b34ac92b151b") failed: Property not found. retry in 556.237707ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_34c7db96-4f54-435b-bea8-b34ac92b151b
11062024-09-01T21:00:04.319ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4091dac0-a803-4f41-a822-732fdb74b5db") failed: Property not found. retry in 560.932442ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4091dac0-a803-4f41-a822-732fdb74b5db
11072024-09-01T21:00:04.553ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cf7f9d18-6fc4-4ee0-b35d-57ed8769bf3d") failed: Property not found. retry in 198.391709ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cf7f9d18-6fc4-4ee0-b35d-57ed8769bf3d
11082024-09-01T21:00:04.559ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_a0ecfdff-5e02-4b4f-863f-6f55a5749ea0: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService16'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::16'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_14681c40-9950-4e30-8f61-ec7a2ca60e1c/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::16'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='38e683d4-6947-43a5-9d75-3c8662c3de4d'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11092024-09-01T21:00:04.564ZINFOSledAgent (ServiceManager): Booting oxz_crucible_a0ecfdff-5e02-4b4f-863f-6f55a5749ea0 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_a0ecfdff-5e02-4b4f-863f-6f55a5749ea0
11102024-09-01T21:00:04.597ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_854f46a4-dab9-4716-b39d-9f437db37063: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService17'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::1a'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_7a19116b-f2a7-4643-8d6b-df9db59b345a/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::1a'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='8a6c96cb-604c-4e53-bae8-c5794148420d'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11112024-09-01T21:00:04.787ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6fe8da63-5815-4e50-aa5f-56ee726aff73") failed: Property not found. retry in 242.538123ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6fe8da63-5815-4e50-aa5f-56ee726aff73
11122024-09-01T21:00:04.804ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_e7ead4f1-17b4-4c7d-843c-7f2e488501a0: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService21'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::11'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::11'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11132024-09-01T21:00:04.810ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_b4673d99-2ce4-485e-a28e-9c3588338668: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService22'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::17'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_8d3c930e-5dba-4325-82af-75dd024bc8cc/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::17'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='69df69ca-62b9-42b1-9423-fb518e7ef93c'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11142024-09-01T21:00:04.852ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_93d2e0a4-500d-430c-b7ae-5ba369a25e48") failed: Property not found. retry in 168.803477ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_93d2e0a4-500d-430c-b7ae-5ba369a25e48
11152024-09-01T21:00:04.903ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_95103d07-7ccb-4d09-b928-56c0fb8e4bb3") failed: Property not found. retry in 770.858554ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_95103d07-7ccb-4d09-b928-56c0fb8e4bb3
11162024-09-01T21:00:04.924ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a7186ed2-a4e0-4db3-9300-6b0a04c483af") failed: Property not found. retry in 765.77226ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a7186ed2-a4e0-4db3-9300-6b0a04c483af
11172024-09-01T21:00:04.943ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_34c7db96-4f54-435b-bea8-b34ac92b151b") failed: Property not found. retry in 963.882524ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_34c7db96-4f54-435b-bea8-b34ac92b151b
11182024-09-01T21:00:04.966ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4091dac0-a803-4f41-a822-732fdb74b5db") failed: Property not found. retry in 802.835895ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4091dac0-a803-4f41-a822-732fdb74b5db
11192024-09-01T21:00:04.988ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cf7f9d18-6fc4-4ee0-b35d-57ed8769bf3d") failed: Property not found. retry in 455.676802ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cf7f9d18-6fc4-4ee0-b35d-57ed8769bf3d
11202024-09-01T21:00:04.988ZINFOSledAgent (ServiceManager): Booting oxz_crucible_854f46a4-dab9-4716-b39d-9f437db37063 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_854f46a4-dab9-4716-b39d-9f437db37063
11212024-09-01T21:00:05.067ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6fe8da63-5815-4e50-aa5f-56ee726aff73") failed: Property not found. retry in 470.244543ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6fe8da63-5815-4e50-aa5f-56ee726aff73
11222024-09-01T21:00:05.068ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_e7ead4f1-17b4-4c7d-843c-7f2e488501a0 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_e7ead4f1-17b4-4c7d-843c-7f2e488501a0
11232024-09-01T21:00:05.081ZINFOSledAgent (ServiceManager): Booting oxz_crucible_b4673d99-2ce4-485e-a28e-9c3588338668 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_b4673d99-2ce4-485e-a28e-9c3588338668
11242024-09-01T21:00:05.390ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_93d2e0a4-500d-430c-b7ae-5ba369a25e48") failed: Property not found. retry in 320.412845ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_93d2e0a4-500d-430c-b7ae-5ba369a25e48
11252024-09-01T21:00:05.453ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_7d3d53d3-b0f8-4024-a319-1703ede11193") failed: Property not found. retry in 870.844212ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_7d3d53d3-b0f8-4024-a319-1703ede11193
11262024-09-01T21:00:05.480ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cf7f9d18-6fc4-4ee0-b35d-57ed8769bf3d") failed: Property not found. retry in 1.129190642s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cf7f9d18-6fc4-4ee0-b35d-57ed8769bf3d
11272024-09-01T21:00:05.569ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6fe8da63-5815-4e50-aa5f-56ee726aff73") failed: Property not found. retry in 822.260795ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6fe8da63-5815-4e50-aa5f-56ee726aff73
11282024-09-01T21:00:05.694ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_95103d07-7ccb-4d09-b928-56c0fb8e4bb3") failed: Property not found. retry in 1.477362576s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_95103d07-7ccb-4d09-b928-56c0fb8e4bb3
11292024-09-01T21:00:05.711ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a7186ed2-a4e0-4db3-9300-6b0a04c483af") failed: Property not found. retry in 551.953838ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a7186ed2-a4e0-4db3-9300-6b0a04c483af
11302024-09-01T21:00:05.726ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a0ecfdff-5e02-4b4f-863f-6f55a5749ea0") failed: Property not found. retry in 65.57854ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a0ecfdff-5e02-4b4f-863f-6f55a5749ea0
11312024-09-01T21:00:05.742ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_93d2e0a4-500d-430c-b7ae-5ba369a25e48") failed: Property not found. retry in 565.194904ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_93d2e0a4-500d-430c-b7ae-5ba369a25e48
11322024-09-01T21:00:05.792ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4091dac0-a803-4f41-a822-732fdb74b5db") failed: Property not found. retry in 1.491156219s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4091dac0-a803-4f41-a822-732fdb74b5db
11332024-09-01T21:00:05.810ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_854f46a4-dab9-4716-b39d-9f437db37063") failed: Property not found. retry in 56.187746ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_854f46a4-dab9-4716-b39d-9f437db37063
11342024-09-01T21:00:05.827ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_e7ead4f1-17b4-4c7d-843c-7f2e488501a0") failed: Property not found. retry in 29.670069ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_e7ead4f1-17b4-4c7d-843c-7f2e488501a0
11352024-09-01T21:00:05.878ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b4673d99-2ce4-485e-a28e-9c3588338668") failed: Property not found. retry in 68.04003ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b4673d99-2ce4-485e-a28e-9c3588338668
11362024-09-01T21:00:05.903ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a0ecfdff-5e02-4b4f-863f-6f55a5749ea0") failed: Property not found. retry in 98.938057ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a0ecfdff-5e02-4b4f-863f-6f55a5749ea0
11372024-09-01T21:00:05.982ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_854f46a4-dab9-4716-b39d-9f437db37063") failed: Property not found. retry in 67.225683ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_854f46a4-dab9-4716-b39d-9f437db37063
11382024-09-01T21:00:06.000ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_e7ead4f1-17b4-4c7d-843c-7f2e488501a0") failed: Property not found. retry in 135.039875ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_e7ead4f1-17b4-4c7d-843c-7f2e488501a0
11392024-09-01T21:00:06.030ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b4673d99-2ce4-485e-a28e-9c3588338668") failed: Property not found. retry in 99.840869ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b4673d99-2ce4-485e-a28e-9c3588338668
11402024-09-01T21:00:06.055ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_34c7db96-4f54-435b-bea8-b34ac92b151b") failed: Property not found. retry in 1.204585706s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_34c7db96-4f54-435b-bea8-b34ac92b151b
11412024-09-01T21:00:06.087ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a0ecfdff-5e02-4b4f-863f-6f55a5749ea0") failed: Property not found. retry in 157.953241ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a0ecfdff-5e02-4b4f-863f-6f55a5749ea0
11422024-09-01T21:00:06.118ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_854f46a4-dab9-4716-b39d-9f437db37063") failed: Property not found. retry in 219.189162ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_854f46a4-dab9-4716-b39d-9f437db37063
11432024-09-01T21:00:06.176ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b4673d99-2ce4-485e-a28e-9c3588338668") failed: Property not found. retry in 118.179502ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b4673d99-2ce4-485e-a28e-9c3588338668
11442024-09-01T21:00:06.210ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_e7ead4f1-17b4-4c7d-843c-7f2e488501a0") failed: Property not found. retry in 283.329643ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_e7ead4f1-17b4-4c7d-843c-7f2e488501a0
11452024-09-01T21:00:06.281ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a0ecfdff-5e02-4b4f-863f-6f55a5749ea0") failed: Property not found. retry in 522.050458ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a0ecfdff-5e02-4b4f-863f-6f55a5749ea0
11462024-09-01T21:00:06.302ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a7186ed2-a4e0-4db3-9300-6b0a04c483af") failed: Property not found. retry in 1.214481519s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a7186ed2-a4e0-4db3-9300-6b0a04c483af
11472024-09-01T21:00:06.329ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b4673d99-2ce4-485e-a28e-9c3588338668") failed: Property not found. retry in 363.644955ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b4673d99-2ce4-485e-a28e-9c3588338668
11482024-09-01T21:00:06.355ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_93d2e0a4-500d-430c-b7ae-5ba369a25e48") failed: Property not found. retry in 589.093726ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_93d2e0a4-500d-430c-b7ae-5ba369a25e48
11492024-09-01T21:00:06.398ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_7d3d53d3-b0f8-4024-a319-1703ede11193") failed: Property not found. retry in 1.013168147s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_7d3d53d3-b0f8-4024-a319-1703ede11193
11502024-09-01T21:00:06.449ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_854f46a4-dab9-4716-b39d-9f437db37063") failed: Property not found. retry in 564.291257ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_854f46a4-dab9-4716-b39d-9f437db37063
11512024-09-01T21:00:06.486ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6fe8da63-5815-4e50-aa5f-56ee726aff73") failed: Property not found. retry in 1.289795227s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6fe8da63-5815-4e50-aa5f-56ee726aff73
11522024-09-01T21:00:06.523ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_e7ead4f1-17b4-4c7d-843c-7f2e488501a0") failed: Property not found. retry in 257.442128ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_e7ead4f1-17b4-4c7d-843c-7f2e488501a0
11532024-09-01T21:00:06.661ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cf7f9d18-6fc4-4ee0-b35d-57ed8769bf3d") failed: Property not found. retry in 762.702494ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cf7f9d18-6fc4-4ee0-b35d-57ed8769bf3d
11542024-09-01T21:00:06.721ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b4673d99-2ce4-485e-a28e-9c3588338668") failed: Property not found. retry in 1.124278846s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b4673d99-2ce4-485e-a28e-9c3588338668
11552024-09-01T21:00:06.806ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_e7ead4f1-17b4-4c7d-843c-7f2e488501a0") failed: Property not found. retry in 1.079232384s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_e7ead4f1-17b4-4c7d-843c-7f2e488501a0
11562024-09-01T21:00:06.827ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a0ecfdff-5e02-4b4f-863f-6f55a5749ea0") failed: Property not found. retry in 619.150248ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a0ecfdff-5e02-4b4f-863f-6f55a5749ea0
11572024-09-01T21:00:06.927ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 70.652540337s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
11582024-09-01T21:00:06.966ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_93d2e0a4-500d-430c-b7ae-5ba369a25e48") failed: Property not found. retry in 1.147421349s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_93d2e0a4-500d-430c-b7ae-5ba369a25e48
11592024-09-01T21:00:07.039ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_854f46a4-dab9-4716-b39d-9f437db37063") failed: Property not found. retry in 887.359125ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_854f46a4-dab9-4716-b39d-9f437db37063
11602024-09-01T21:00:07.190ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_95103d07-7ccb-4d09-b928-56c0fb8e4bb3") failed: Property not found. retry in 1.410647661s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_95103d07-7ccb-4d09-b928-56c0fb8e4bb3
11612024-09-01T21:00:07.276ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_34c7db96-4f54-435b-bea8-b34ac92b151b") failed: Property not found. retry in 743.989633ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_34c7db96-4f54-435b-bea8-b34ac92b151b
11622024-09-01T21:00:07.300ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4091dac0-a803-4f41-a822-732fdb74b5db") failed: Property not found. retry in 624.158739ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4091dac0-a803-4f41-a822-732fdb74b5db
11632024-09-01T21:00:07.427ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_7d3d53d3-b0f8-4024-a319-1703ede11193") failed: Property not found. retry in 1.157686118s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_7d3d53d3-b0f8-4024-a319-1703ede11193
11642024-09-01T21:00:07.443ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cf7f9d18-6fc4-4ee0-b35d-57ed8769bf3d") failed: Property not found. retry in 784.522587ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cf7f9d18-6fc4-4ee0-b35d-57ed8769bf3d
11652024-09-01T21:00:07.468ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a0ecfdff-5e02-4b4f-863f-6f55a5749ea0") failed: Property not found. retry in 1.237746974s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a0ecfdff-5e02-4b4f-863f-6f55a5749ea0
11662024-09-01T21:00:07.534ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a7186ed2-a4e0-4db3-9300-6b0a04c483af") failed: Property not found. retry in 548.92826ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a7186ed2-a4e0-4db3-9300-6b0a04c483af
11672024-09-01T21:00:07.544ZINFOSledAgent (ServiceManager): Profile for oxz_external_dns_e1c0d41c-ad07-4854-9798-d69fb32da3c7: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService28'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::9'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte4'/> <propval type="astring" name="gateway" value='172.30.1.1'/> <propval type="astring" name="ip" value='172.30.1.6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/external_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::9]:5353'/> <propval type="astring" name="dns_address" value='172.30.1.6:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11682024-09-01T21:00:07.545ZINFOSledAgent (ServiceManager): Booting oxz_external_dns_e1c0d41c-ad07-4854-9798-d69fb32da3c7 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_external_dns_e1c0d41c-ad07-4854-9798-d69fb32da3c7
11692024-09-01T21:00:07.794ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6fe8da63-5815-4e50-aa5f-56ee726aff73") failed: Property not found. retry in 759.611629ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6fe8da63-5815-4e50-aa5f-56ee726aff73
11702024-09-01T21:00:07.845ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_e1c0d41c-ad07-4854-9798-d69fb32da3c7") failed: Property not found. retry in 42.500122ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_e1c0d41c-ad07-4854-9798-d69fb32da3c7
11712024-09-01T21:00:07.871ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b4673d99-2ce4-485e-a28e-9c3588338668") failed: Property not found. retry in 1.06458285s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b4673d99-2ce4-485e-a28e-9c3588338668
11722024-09-01T21:00:07.905ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_e7ead4f1-17b4-4c7d-843c-7f2e488501a0") failed: Property not found. retry in 886.702933ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_e7ead4f1-17b4-4c7d-843c-7f2e488501a0
11732024-09-01T21:00:07.921ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_e1c0d41c-ad07-4854-9798-d69fb32da3c7") failed: Property not found. retry in 88.417462ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_e1c0d41c-ad07-4854-9798-d69fb32da3c7
11742024-09-01T21:00:07.941ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4091dac0-a803-4f41-a822-732fdb74b5db") failed: Property not found. retry in 968.032875ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4091dac0-a803-4f41-a822-732fdb74b5db
11752024-09-01T21:00:07.960ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_854f46a4-dab9-4716-b39d-9f437db37063") failed: Property not found. retry in 1.19088427s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_854f46a4-dab9-4716-b39d-9f437db37063
11762024-09-01T21:00:08.025ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_e1c0d41c-ad07-4854-9798-d69fb32da3c7") failed: Property not found. retry in 277.541609ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_e1c0d41c-ad07-4854-9798-d69fb32da3c7
11772024-09-01T21:00:08.041ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_34c7db96-4f54-435b-bea8-b34ac92b151b") failed: Property not found. retry in 739.293195ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_34c7db96-4f54-435b-bea8-b34ac92b151b
11782024-09-01T21:00:08.103ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a7186ed2-a4e0-4db3-9300-6b0a04c483af") failed: Property not found. retry in 917.858877ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a7186ed2-a4e0-4db3-9300-6b0a04c483af
11792024-09-01T21:00:08.142ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_93d2e0a4-500d-430c-b7ae-5ba369a25e48") failed: Property not found. retry in 1.005655108s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_93d2e0a4-500d-430c-b7ae-5ba369a25e48
11802024-09-01T21:00:08.258ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cf7f9d18-6fc4-4ee0-b35d-57ed8769bf3d") failed: Property not found. retry in 927.343339ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cf7f9d18-6fc4-4ee0-b35d-57ed8769bf3d
11812024-09-01T21:00:08.330ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_e1c0d41c-ad07-4854-9798-d69fb32da3c7") failed: Property not found. retry in 574.354821ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_e1c0d41c-ad07-4854-9798-d69fb32da3c7
11822024-09-01T21:00:08.331ZINFOSledAgent (ServiceManager): Profile for oxz_external_dns_d0653118-7c84-43ff-9e70-8fe4cb6a2a64: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService24'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::8'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte2'/> <propval type="astring" name="gateway" value='172.30.1.1'/> <propval type="astring" name="ip" value='172.30.1.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/external_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::8]:5353'/> <propval type="astring" name="dns_address" value='172.30.1.5:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11832024-09-01T21:00:08.332ZINFOSledAgent (ServiceManager): Booting oxz_external_dns_d0653118-7c84-43ff-9e70-8fe4cb6a2a64 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_external_dns_d0653118-7c84-43ff-9e70-8fe4cb6a2a64
11842024-09-01T21:00:08.567ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6fe8da63-5815-4e50-aa5f-56ee726aff73") failed: Property not found. retry in 831.295684ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6fe8da63-5815-4e50-aa5f-56ee726aff73
11852024-09-01T21:00:08.599ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_7d3d53d3-b0f8-4024-a319-1703ede11193") failed: Property not found. retry in 867.744178ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_7d3d53d3-b0f8-4024-a319-1703ede11193
11862024-09-01T21:00:08.617ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_95103d07-7ccb-4d09-b928-56c0fb8e4bb3") failed: Property not found. retry in 1.213352206s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_95103d07-7ccb-4d09-b928-56c0fb8e4bb3
11872024-09-01T21:00:08.629ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d0653118-7c84-43ff-9e70-8fe4cb6a2a64") failed: Property not found. retry in 60.986011ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d0653118-7c84-43ff-9e70-8fe4cb6a2a64
11882024-09-01T21:00:08.636ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_0da3314b-38b4-4938-8831-8ec6d7490158: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService19'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::12'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_067f6720-0053-47f5-be0b-12fc815674a2/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::12'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='8ce58f6e-6722-4c86-b2b1-f461491d8b59'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11892024-09-01T21:00:08.637ZINFOSledAgent (ServiceManager): Booting oxz_crucible_0da3314b-38b4-4938-8831-8ec6d7490158 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_0da3314b-38b4-4938-8831-8ec6d7490158
11902024-09-01T21:00:08.710ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d0653118-7c84-43ff-9e70-8fe4cb6a2a64") failed: Property not found. retry in 136.526386ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d0653118-7c84-43ff-9e70-8fe4cb6a2a64
11912024-09-01T21:00:08.734ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a0ecfdff-5e02-4b4f-863f-6f55a5749ea0") failed: Property not found. retry in 509.5021ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a0ecfdff-5e02-4b4f-863f-6f55a5749ea0
11922024-09-01T21:00:08.805ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_34c7db96-4f54-435b-bea8-b34ac92b151b") failed: Property not found. retry in 968.448626ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_34c7db96-4f54-435b-bea8-b34ac92b151b
11932024-09-01T21:00:08.828ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_e7ead4f1-17b4-4c7d-843c-7f2e488501a0") failed: Property not found. retry in 515.19793ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_e7ead4f1-17b4-4c7d-843c-7f2e488501a0
11942024-09-01T21:00:08.864ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d0653118-7c84-43ff-9e70-8fe4cb6a2a64") failed: Property not found. retry in 244.984898ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d0653118-7c84-43ff-9e70-8fe4cb6a2a64
11952024-09-01T21:00:08.933ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_e1c0d41c-ad07-4854-9798-d69fb32da3c7") failed: Property not found. retry in 1.024244439s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_e1c0d41c-ad07-4854-9798-d69fb32da3c7
11962024-09-01T21:00:08.953ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4091dac0-a803-4f41-a822-732fdb74b5db") failed: Property not found. retry in 878.71844ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4091dac0-a803-4f41-a822-732fdb74b5db
11972024-09-01T21:00:08.971ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b4673d99-2ce4-485e-a28e-9c3588338668") failed: Property not found. retry in 1.095030839s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b4673d99-2ce4-485e-a28e-9c3588338668
11982024-09-01T21:00:08.989ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_e6ddac78-e9f2-47ae-8121-95ee98f70ed8: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService25'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::a'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte1'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11992024-09-01T21:00:08.989ZINFOSledAgent (ServiceManager): Booting oxz_nexus_e6ddac78-e9f2-47ae-8121-95ee98f70ed8 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_e6ddac78-e9f2-47ae-8121-95ee98f70ed8
12002024-09-01T21:00:09.044ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a7186ed2-a4e0-4db3-9300-6b0a04c483af") failed: Property not found. retry in 1.200253563s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a7186ed2-a4e0-4db3-9300-6b0a04c483af
12012024-09-01T21:00:09.045ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_f9b91117-5154-4cc0-9de9-2e9efaaad0e6: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService27'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::c'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte3'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.7'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
12022024-09-01T21:00:09.045ZINFOSledAgent (ServiceManager): Booting oxz_nexus_f9b91117-5154-4cc0-9de9-2e9efaaad0e6 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_f9b91117-5154-4cc0-9de9-2e9efaaad0e6
12032024-09-01T21:00:09.205ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d0653118-7c84-43ff-9e70-8fe4cb6a2a64") failed: Property not found. retry in 399.831715ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d0653118-7c84-43ff-9e70-8fe4cb6a2a64
12042024-09-01T21:00:09.229ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cf7f9d18-6fc4-4ee0-b35d-57ed8769bf3d") failed: Property not found. retry in 949.884271ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cf7f9d18-6fc4-4ee0-b35d-57ed8769bf3d
12052024-09-01T21:00:09.250ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_854f46a4-dab9-4716-b39d-9f437db37063") failed: Property not found. retry in 704.458217ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_854f46a4-dab9-4716-b39d-9f437db37063
12062024-09-01T21:00:09.268ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_93d2e0a4-500d-430c-b7ae-5ba369a25e48") failed: Property not found. retry in 1.162616043s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_93d2e0a4-500d-430c-b7ae-5ba369a25e48
12072024-09-01T21:00:09.285ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a0ecfdff-5e02-4b4f-863f-6f55a5749ea0") failed: Property not found. retry in 1.017879029s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a0ecfdff-5e02-4b4f-863f-6f55a5749ea0
12082024-09-01T21:00:09.333ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e6ddac78-e9f2-47ae-8121-95ee98f70ed8") failed: Property not found. retry in 66.548984ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e6ddac78-e9f2-47ae-8121-95ee98f70ed8
12092024-09-01T21:00:09.363ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_e7ead4f1-17b4-4c7d-843c-7f2e488501a0") failed: Property not found. retry in 566.756225ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_e7ead4f1-17b4-4c7d-843c-7f2e488501a0
12102024-09-01T21:00:09.420ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6fe8da63-5815-4e50-aa5f-56ee726aff73") failed: Property not found. retry in 611.343501ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6fe8da63-5815-4e50-aa5f-56ee726aff73
12112024-09-01T21:00:09.440ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e6ddac78-e9f2-47ae-8121-95ee98f70ed8") failed: Property not found. retry in 149.263004ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e6ddac78-e9f2-47ae-8121-95ee98f70ed8
12122024-09-01T21:00:09.538ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_7d3d53d3-b0f8-4024-a319-1703ede11193") failed: Property not found. retry in 1.026120233s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_7d3d53d3-b0f8-4024-a319-1703ede11193
12132024-09-01T21:00:09.651ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_f9b91117-5154-4cc0-9de9-2e9efaaad0e6") failed: Property not found. retry in 65.632671ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_f9b91117-5154-4cc0-9de9-2e9efaaad0e6
12142024-09-01T21:00:09.684ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d0653118-7c84-43ff-9e70-8fe4cb6a2a64") failed: Property not found. retry in 659.628239ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d0653118-7c84-43ff-9e70-8fe4cb6a2a64
12152024-09-01T21:00:09.818ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e6ddac78-e9f2-47ae-8121-95ee98f70ed8") failed: Property not found. retry in 109.000083ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e6ddac78-e9f2-47ae-8121-95ee98f70ed8
12162024-09-01T21:00:09.833ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_f9b91117-5154-4cc0-9de9-2e9efaaad0e6") failed: Property not found. retry in 135.602525ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_f9b91117-5154-4cc0-9de9-2e9efaaad0e6
12172024-09-01T21:00:09.846ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_34c7db96-4f54-435b-bea8-b34ac92b151b") failed: Property not found. retry in 910.353261ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_34c7db96-4f54-435b-bea8-b34ac92b151b
12182024-09-01T21:00:09.859ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4091dac0-a803-4f41-a822-732fdb74b5db") failed: Property not found. retry in 1.430497563s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4091dac0-a803-4f41-a822-732fdb74b5db
12192024-09-01T21:00:09.870ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_95103d07-7ccb-4d09-b928-56c0fb8e4bb3") failed: Property not found. retry in 1.287182297s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_95103d07-7ccb-4d09-b928-56c0fb8e4bb3
12202024-09-01T21:00:09.939ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e6ddac78-e9f2-47ae-8121-95ee98f70ed8") failed: Property not found. retry in 270.740488ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e6ddac78-e9f2-47ae-8121-95ee98f70ed8
12212024-09-01T21:00:09.950ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_e7ead4f1-17b4-4c7d-843c-7f2e488501a0") failed: Property not found. retry in 1.14002089s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_e7ead4f1-17b4-4c7d-843c-7f2e488501a0
12222024-09-01T21:00:09.967ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_854f46a4-dab9-4716-b39d-9f437db37063") failed: Property not found. retry in 745.049206ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_854f46a4-dab9-4716-b39d-9f437db37063
12232024-09-01T21:00:09.979ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_e1c0d41c-ad07-4854-9798-d69fb32da3c7") failed: Property not found. retry in 775.646247ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_e1c0d41c-ad07-4854-9798-d69fb32da3c7
12242024-09-01T21:00:09.996ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_f9b91117-5154-4cc0-9de9-2e9efaaad0e6") failed: Property not found. retry in 291.099828ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_f9b91117-5154-4cc0-9de9-2e9efaaad0e6
12252024-09-01T21:00:10.043ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6fe8da63-5815-4e50-aa5f-56ee726aff73") failed: Property not found. retry in 846.010527ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6fe8da63-5815-4e50-aa5f-56ee726aff73
12262024-09-01T21:00:10.078ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b4673d99-2ce4-485e-a28e-9c3588338668") failed: Property not found. retry in 855.917397ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b4673d99-2ce4-485e-a28e-9c3588338668
12272024-09-01T21:00:10.193ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cf7f9d18-6fc4-4ee0-b35d-57ed8769bf3d") failed: Property not found. retry in 761.499362ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cf7f9d18-6fc4-4ee0-b35d-57ed8769bf3d
12282024-09-01T21:00:10.222ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e6ddac78-e9f2-47ae-8121-95ee98f70ed8") failed: Property not found. retry in 790.059189ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e6ddac78-e9f2-47ae-8121-95ee98f70ed8
12292024-09-01T21:00:10.259ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a7186ed2-a4e0-4db3-9300-6b0a04c483af") failed: Property not found. retry in 1.118520278s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a7186ed2-a4e0-4db3-9300-6b0a04c483af
12302024-09-01T21:00:10.298ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_f9b91117-5154-4cc0-9de9-2e9efaaad0e6") failed: Property not found. retry in 212.710247ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_f9b91117-5154-4cc0-9de9-2e9efaaad0e6
12312024-09-01T21:00:10.315ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a0ecfdff-5e02-4b4f-863f-6f55a5749ea0") failed: Property not found. retry in 1.487122934s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a0ecfdff-5e02-4b4f-863f-6f55a5749ea0
12322024-09-01T21:00:10.361ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d0653118-7c84-43ff-9e70-8fe4cb6a2a64") failed: Property not found. retry in 1.21870476s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d0653118-7c84-43ff-9e70-8fe4cb6a2a64
12332024-09-01T21:00:10.444ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_93d2e0a4-500d-430c-b7ae-5ba369a25e48") failed: Property not found. retry in 1.346309103s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_93d2e0a4-500d-430c-b7ae-5ba369a25e48
12342024-09-01T21:00:10.474ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0da3314b-38b4-4938-8831-8ec6d7490158") failed: Property not found. retry in 66.116349ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0da3314b-38b4-4938-8831-8ec6d7490158
12352024-09-01T21:00:10.528ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_f9b91117-5154-4cc0-9de9-2e9efaaad0e6") failed: Property not found. retry in 1.068993083s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_f9b91117-5154-4cc0-9de9-2e9efaaad0e6
12362024-09-01T21:00:10.556ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0da3314b-38b4-4938-8831-8ec6d7490158") failed: Property not found. retry in 132.052265ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0da3314b-38b4-4938-8831-8ec6d7490158
12372024-09-01T21:00:10.596ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_7d3d53d3-b0f8-4024-a319-1703ede11193") failed: Property not found. retry in 519.704083ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_7d3d53d3-b0f8-4024-a319-1703ede11193
12382024-09-01T21:00:10.701ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0da3314b-38b4-4938-8831-8ec6d7490158") failed: Property not found. retry in 225.901699ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0da3314b-38b4-4938-8831-8ec6d7490158
12392024-09-01T21:00:10.724ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_854f46a4-dab9-4716-b39d-9f437db37063") failed: Property not found. retry in 502.188039ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_854f46a4-dab9-4716-b39d-9f437db37063
12402024-09-01T21:00:10.771ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_e1c0d41c-ad07-4854-9798-d69fb32da3c7") failed: Property not found. retry in 1.330406467s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_e1c0d41c-ad07-4854-9798-d69fb32da3c7
12412024-09-01T21:00:10.785ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_34c7db96-4f54-435b-bea8-b34ac92b151b") failed: Property not found. retry in 1.472651722s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_34c7db96-4f54-435b-bea8-b34ac92b151b
12422024-09-01T21:00:10.906ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6fe8da63-5815-4e50-aa5f-56ee726aff73") failed: Property not found. retry in 1.451607821s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6fe8da63-5815-4e50-aa5f-56ee726aff73
12432024-09-01T21:00:10.947ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0da3314b-38b4-4938-8831-8ec6d7490158") failed: Property not found. retry in 284.713724ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0da3314b-38b4-4938-8831-8ec6d7490158
12442024-09-01T21:00:10.962ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b4673d99-2ce4-485e-a28e-9c3588338668") failed: Property not found. retry in 663.50294ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b4673d99-2ce4-485e-a28e-9c3588338668
12452024-09-01T21:00:10.978ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cf7f9d18-6fc4-4ee0-b35d-57ed8769bf3d") failed: Property not found. retry in 1.257676211s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cf7f9d18-6fc4-4ee0-b35d-57ed8769bf3d
12462024-09-01T21:00:11.028ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e6ddac78-e9f2-47ae-8121-95ee98f70ed8") failed: Property not found. retry in 1.352361508s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e6ddac78-e9f2-47ae-8121-95ee98f70ed8
12472024-09-01T21:00:11.051ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_fcd4de7a-43df-4ccf-a747-0c745f76751c: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService26'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::b'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte5'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
12482024-09-01T21:00:11.051ZINFOSledAgent (ServiceManager): Booting oxz_nexus_fcd4de7a-43df-4ccf-a747-0c745f76751c zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_fcd4de7a-43df-4ccf-a747-0c745f76751c
12492024-09-01T21:00:11.104ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_e7ead4f1-17b4-4c7d-843c-7f2e488501a0") failed: Property not found. retry in 874.826554ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_e7ead4f1-17b4-4c7d-843c-7f2e488501a0
12502024-09-01T21:00:11.129ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_7d3d53d3-b0f8-4024-a319-1703ede11193") failed: Property not found. retry in 784.272511ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_7d3d53d3-b0f8-4024-a319-1703ede11193
12512024-09-01T21:00:11.174ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_95103d07-7ccb-4d09-b928-56c0fb8e4bb3") failed: Property not found. retry in 630.574266ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_95103d07-7ccb-4d09-b928-56c0fb8e4bb3
12522024-09-01T21:00:11.240ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_854f46a4-dab9-4716-b39d-9f437db37063") failed: Property not found. retry in 1.286622737s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_854f46a4-dab9-4716-b39d-9f437db37063
12532024-09-01T21:00:11.255ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0da3314b-38b4-4938-8831-8ec6d7490158") failed: Property not found. retry in 1.013964264s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0da3314b-38b4-4938-8831-8ec6d7490158
12542024-09-01T21:00:11.299ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_fcd4de7a-43df-4ccf-a747-0c745f76751c") failed: Property not found. retry in 43.559238ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_fcd4de7a-43df-4ccf-a747-0c745f76751c
12552024-09-01T21:00:11.313ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4091dac0-a803-4f41-a822-732fdb74b5db") failed: Property not found. retry in 1.403432463s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4091dac0-a803-4f41-a822-732fdb74b5db
12562024-09-01T21:00:11.355ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_fcd4de7a-43df-4ccf-a747-0c745f76751c") failed: Property not found. retry in 114.92292ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_fcd4de7a-43df-4ccf-a747-0c745f76751c
12572024-09-01T21:00:11.392ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a7186ed2-a4e0-4db3-9300-6b0a04c483af") failed: Property not found. retry in 1.226745745s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a7186ed2-a4e0-4db3-9300-6b0a04c483af
12582024-09-01T21:00:11.467ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:49615
12592024-09-01T21:00:11.509ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 24037 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:49615 req_id = 9ee42e01-4f82-4372-bf7d-ecc911b8ecae response_code = 200 uri = /baseboard
12602024-09-01T21:00:11.520ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_fcd4de7a-43df-4ccf-a747-0c745f76751c") failed: Property not found. retry in 239.50632ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_fcd4de7a-43df-4ccf-a747-0c745f76751c
12612024-09-01T21:00:11.596ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d0653118-7c84-43ff-9e70-8fe4cb6a2a64") failed: Property not found. retry in 1.302096009s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d0653118-7c84-43ff-9e70-8fe4cb6a2a64
12622024-09-01T21:00:11.610ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_f9b91117-5154-4cc0-9de9-2e9efaaad0e6") failed: Property not found. retry in 827.251783ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_f9b91117-5154-4cc0-9de9-2e9efaaad0e6
12632024-09-01T21:00:11.639ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b4673d99-2ce4-485e-a28e-9c3588338668") failed: Property not found. retry in 1.468557009s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b4673d99-2ce4-485e-a28e-9c3588338668
12642024-09-01T21:00:11.778ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_fcd4de7a-43df-4ccf-a747-0c745f76751c") failed: Property not found. retry in 496.524677ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_fcd4de7a-43df-4ccf-a747-0c745f76751c
12652024-09-01T21:00:11.807ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_93d2e0a4-500d-430c-b7ae-5ba369a25e48") failed: Property not found. retry in 568.801859ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_93d2e0a4-500d-430c-b7ae-5ba369a25e48
12662024-09-01T21:00:11.821ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_95103d07-7ccb-4d09-b928-56c0fb8e4bb3") failed: Property not found. retry in 984.802357ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_95103d07-7ccb-4d09-b928-56c0fb8e4bb3
12672024-09-01T21:00:11.834ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a0ecfdff-5e02-4b4f-863f-6f55a5749ea0") failed: Property not found. retry in 1.132951163s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a0ecfdff-5e02-4b4f-863f-6f55a5749ea0
12682024-09-01T21:00:11.930ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_7d3d53d3-b0f8-4024-a319-1703ede11193") failed: Property not found. retry in 1.147006552s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_7d3d53d3-b0f8-4024-a319-1703ede11193
12692024-09-01T21:00:11.996ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_e7ead4f1-17b4-4c7d-843c-7f2e488501a0") failed: Property not found. retry in 519.690754ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_e7ead4f1-17b4-4c7d-843c-7f2e488501a0
12702024-09-01T21:00:12.116ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_e1c0d41c-ad07-4854-9798-d69fb32da3c7") failed: Property not found. retry in 992.934728ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_e1c0d41c-ad07-4854-9798-d69fb32da3c7
12712024-09-01T21:00:12.254ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cf7f9d18-6fc4-4ee0-b35d-57ed8769bf3d") failed: Property not found. retry in 1.056586571s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cf7f9d18-6fc4-4ee0-b35d-57ed8769bf3d
12722024-09-01T21:00:12.272ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_34c7db96-4f54-435b-bea8-b34ac92b151b") failed: Property not found. retry in 889.042619ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_34c7db96-4f54-435b-bea8-b34ac92b151b
12732024-09-01T21:00:12.287ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0da3314b-38b4-4938-8831-8ec6d7490158") failed: Property not found. retry in 1.369614216s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0da3314b-38b4-4938-8831-8ec6d7490158
12742024-09-01T21:00:12.302ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_fcd4de7a-43df-4ccf-a747-0c745f76751c") failed: Property not found. retry in 1.197757931s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_fcd4de7a-43df-4ccf-a747-0c745f76751c
12752024-09-01T21:00:12.374ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6fe8da63-5815-4e50-aa5f-56ee726aff73") failed: Property not found. retry in 1.113644243s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6fe8da63-5815-4e50-aa5f-56ee726aff73
12762024-09-01T21:00:12.392ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_93d2e0a4-500d-430c-b7ae-5ba369a25e48") failed: Property not found. retry in 1.267725373s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_93d2e0a4-500d-430c-b7ae-5ba369a25e48
12772024-09-01T21:00:12.408ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e6ddac78-e9f2-47ae-8121-95ee98f70ed8") failed: Property not found. retry in 1.363552771s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e6ddac78-e9f2-47ae-8121-95ee98f70ed8
12782024-09-01T21:00:12.453ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_f9b91117-5154-4cc0-9de9-2e9efaaad0e6") failed: Property not found. retry in 1.105476703s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_f9b91117-5154-4cc0-9de9-2e9efaaad0e6
12792024-09-01T21:00:12.531ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_e7ead4f1-17b4-4c7d-843c-7f2e488501a0") failed: Property not found. retry in 1.472735137s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_e7ead4f1-17b4-4c7d-843c-7f2e488501a0
12802024-09-01T21:00:12.547ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_854f46a4-dab9-4716-b39d-9f437db37063") failed: Property not found. retry in 906.940177ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_854f46a4-dab9-4716-b39d-9f437db37063
12812024-09-01T21:00:12.633ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a7186ed2-a4e0-4db3-9300-6b0a04c483af") failed: Property not found. retry in 558.434173ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a7186ed2-a4e0-4db3-9300-6b0a04c483af
12822024-09-01T21:00:12.733ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4091dac0-a803-4f41-a822-732fdb74b5db") failed: Property not found. retry in 1.213676627s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4091dac0-a803-4f41-a822-732fdb74b5db
12832024-09-01T21:00:12.826ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_95103d07-7ccb-4d09-b928-56c0fb8e4bb3") failed: Property not found. retry in 1.12257415s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_95103d07-7ccb-4d09-b928-56c0fb8e4bb3
12842024-09-01T21:00:12.914ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d0653118-7c84-43ff-9e70-8fe4cb6a2a64") failed: Property not found. retry in 514.778506ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d0653118-7c84-43ff-9e70-8fe4cb6a2a64
12852024-09-01T21:00:12.917ZINFOSledAgent (ServiceManager): Profile for oxz_clickhouse_03bcf173-e5c2-4545-ac63-b6f41f5cbdda: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService18'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::e'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/clickhouse"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::e'/> <propval type="astring" name="listen_port" value='8123'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/clickhouse-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="clickhouse_address" value='[fd00:1122:3344:101::e]:8123'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::e]:8888'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
12862024-09-01T21:00:12.918ZINFOSledAgent (ServiceManager): Booting oxz_clickhouse_03bcf173-e5c2-4545-ac63-b6f41f5cbdda zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_clickhouse_03bcf173-e5c2-4545-ac63-b6f41f5cbdda
12872024-09-01T21:00:12.983ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a0ecfdff-5e02-4b4f-863f-6f55a5749ea0") failed: Property not found. retry in 1.103358098s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a0ecfdff-5e02-4b4f-863f-6f55a5749ea0
12882024-09-01T21:00:13.094ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_7d3d53d3-b0f8-4024-a319-1703ede11193") failed: Property not found. retry in 935.613435ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_7d3d53d3-b0f8-4024-a319-1703ede11193
12892024-09-01T21:00:13.121ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b4673d99-2ce4-485e-a28e-9c3588338668") failed: Property not found. retry in 1.377228229s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b4673d99-2ce4-485e-a28e-9c3588338668
12902024-09-01T21:00:13.134ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_e1c0d41c-ad07-4854-9798-d69fb32da3c7") failed: Property not found. retry in 887.039486ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_e1c0d41c-ad07-4854-9798-d69fb32da3c7
12912024-09-01T21:00:13.173ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_34c7db96-4f54-435b-bea8-b34ac92b151b") failed: Property not found. retry in 1.433256819s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_34c7db96-4f54-435b-bea8-b34ac92b151b
12922024-09-01T21:00:13.183ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_03bcf173-e5c2-4545-ac63-b6f41f5cbdda") failed: Property not found. retry in 55.398827ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_03bcf173-e5c2-4545-ac63-b6f41f5cbdda
12932024-09-01T21:00:13.203ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a7186ed2-a4e0-4db3-9300-6b0a04c483af") failed: Property not found. retry in 666.217999ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a7186ed2-a4e0-4db3-9300-6b0a04c483af
12942024-09-01T21:00:13.252ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_03bcf173-e5c2-4545-ac63-b6f41f5cbdda") failed: Property not found. retry in 94.943746ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_03bcf173-e5c2-4545-ac63-b6f41f5cbdda
12952024-09-01T21:00:13.345ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cf7f9d18-6fc4-4ee0-b35d-57ed8769bf3d") failed: Property not found. retry in 921.108423ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cf7f9d18-6fc4-4ee0-b35d-57ed8769bf3d
12962024-09-01T21:00:13.401ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_03bcf173-e5c2-4545-ac63-b6f41f5cbdda") failed: Property not found. retry in 299.363198ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_03bcf173-e5c2-4545-ac63-b6f41f5cbdda
12972024-09-01T21:00:13.446ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d0653118-7c84-43ff-9e70-8fe4cb6a2a64") failed: Property not found. retry in 799.463808ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d0653118-7c84-43ff-9e70-8fe4cb6a2a64
12982024-09-01T21:00:13.468ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_854f46a4-dab9-4716-b39d-9f437db37063") failed: Property not found. retry in 949.354515ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_854f46a4-dab9-4716-b39d-9f437db37063
12992024-09-01T21:00:13.504ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6fe8da63-5815-4e50-aa5f-56ee726aff73") failed: Property not found. retry in 1.083547669s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6fe8da63-5815-4e50-aa5f-56ee726aff73
13002024-09-01T21:00:13.522ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_fcd4de7a-43df-4ccf-a747-0c745f76751c") failed: Property not found. retry in 1.326437298s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_fcd4de7a-43df-4ccf-a747-0c745f76751c
13012024-09-01T21:00:13.574ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_f9b91117-5154-4cc0-9de9-2e9efaaad0e6") failed: Property not found. retry in 608.198621ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_f9b91117-5154-4cc0-9de9-2e9efaaad0e6
13022024-09-01T21:00:13.669ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0da3314b-38b4-4938-8831-8ec6d7490158") failed: Property not found. retry in 1.121601789s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0da3314b-38b4-4938-8831-8ec6d7490158
13032024-09-01T21:00:13.684ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_93d2e0a4-500d-430c-b7ae-5ba369a25e48") failed: Property not found. retry in 1.206055997s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_93d2e0a4-500d-430c-b7ae-5ba369a25e48
13042024-09-01T21:00:13.719ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_03bcf173-e5c2-4545-ac63-b6f41f5cbdda") failed: Property not found. retry in 456.097035ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_03bcf173-e5c2-4545-ac63-b6f41f5cbdda
13052024-09-01T21:00:13.789ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e6ddac78-e9f2-47ae-8121-95ee98f70ed8") failed: Property not found. retry in 1.295635318s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e6ddac78-e9f2-47ae-8121-95ee98f70ed8
13062024-09-01T21:00:13.889ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a7186ed2-a4e0-4db3-9300-6b0a04c483af") failed: Property not found. retry in 689.374674ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a7186ed2-a4e0-4db3-9300-6b0a04c483af
13072024-09-01T21:00:13.964ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4091dac0-a803-4f41-a822-732fdb74b5db") failed: Property not found. retry in 705.571009ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4091dac0-a803-4f41-a822-732fdb74b5db
13082024-09-01T21:00:13.980ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_95103d07-7ccb-4d09-b928-56c0fb8e4bb3") failed: Property not found. retry in 795.797078ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_95103d07-7ccb-4d09-b928-56c0fb8e4bb3
13092024-09-01T21:00:14.019ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_e7ead4f1-17b4-4c7d-843c-7f2e488501a0") failed: Property not found. retry in 1.436988145s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_e7ead4f1-17b4-4c7d-843c-7f2e488501a0
13102024-09-01T21:00:14.037ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_e1c0d41c-ad07-4854-9798-d69fb32da3c7") failed: Property not found. retry in 993.742486ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_e1c0d41c-ad07-4854-9798-d69fb32da3c7
13112024-09-01T21:00:14.051ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_7d3d53d3-b0f8-4024-a319-1703ede11193") failed: Property not found. retry in 814.932612ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_7d3d53d3-b0f8-4024-a319-1703ede11193
13122024-09-01T21:00:14.103ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a0ecfdff-5e02-4b4f-863f-6f55a5749ea0") failed: Property not found. retry in 1.467545701s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a0ecfdff-5e02-4b4f-863f-6f55a5749ea0
13132024-09-01T21:00:14.192ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_03bcf173-e5c2-4545-ac63-b6f41f5cbdda") failed: Property not found. retry in 580.8238ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_03bcf173-e5c2-4545-ac63-b6f41f5cbdda
13142024-09-01T21:00:14.206ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_f9b91117-5154-4cc0-9de9-2e9efaaad0e6") failed: Property not found. retry in 916.079225ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_f9b91117-5154-4cc0-9de9-2e9efaaad0e6
13152024-09-01T21:00:14.263ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d0653118-7c84-43ff-9e70-8fe4cb6a2a64") failed: Property not found. retry in 527.87824ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d0653118-7c84-43ff-9e70-8fe4cb6a2a64
13162024-09-01T21:00:14.281ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cf7f9d18-6fc4-4ee0-b35d-57ed8769bf3d") failed: Property not found. retry in 883.050619ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cf7f9d18-6fc4-4ee0-b35d-57ed8769bf3d
13172024-09-01T21:00:14.436ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_854f46a4-dab9-4716-b39d-9f437db37063") failed: Property not found. retry in 1.343299071s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_854f46a4-dab9-4716-b39d-9f437db37063
13182024-09-01T21:00:14.520ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b4673d99-2ce4-485e-a28e-9c3588338668") failed: Property not found. retry in 1.337167552s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b4673d99-2ce4-485e-a28e-9c3588338668
13192024-09-01T21:00:14.595ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a7186ed2-a4e0-4db3-9300-6b0a04c483af") failed: Property not found. retry in 1.054635265s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a7186ed2-a4e0-4db3-9300-6b0a04c483af
13202024-09-01T21:00:14.611ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6fe8da63-5815-4e50-aa5f-56ee726aff73") failed: Property not found. retry in 870.05651ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6fe8da63-5815-4e50-aa5f-56ee726aff73
13212024-09-01T21:00:14.701ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4091dac0-a803-4f41-a822-732fdb74b5db") failed: Property not found. retry in 653.658342ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4091dac0-a803-4f41-a822-732fdb74b5db
13222024-09-01T21:00:14.790ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_03bcf173-e5c2-4545-ac63-b6f41f5cbdda") failed: Property not found. retry in 1.367378797s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_03bcf173-e5c2-4545-ac63-b6f41f5cbdda
13232024-09-01T21:00:14.806ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_95103d07-7ccb-4d09-b928-56c0fb8e4bb3") failed: Property not found. retry in 655.033315ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_95103d07-7ccb-4d09-b928-56c0fb8e4bb3
13242024-09-01T21:00:14.823ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d0653118-7c84-43ff-9e70-8fe4cb6a2a64") failed: Property not found. retry in 747.49702ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d0653118-7c84-43ff-9e70-8fe4cb6a2a64
13252024-09-01T21:00:14.837ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0da3314b-38b4-4938-8831-8ec6d7490158") failed: Property not found. retry in 619.570281ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0da3314b-38b4-4938-8831-8ec6d7490158
13262024-09-01T21:00:14.866ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_fcd4de7a-43df-4ccf-a747-0c745f76751c") failed: Property not found. retry in 809.041813ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_fcd4de7a-43df-4ccf-a747-0c745f76751c
13272024-09-01T21:00:14.883ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_7d3d53d3-b0f8-4024-a319-1703ede11193") failed: Property not found. retry in 804.513819ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_7d3d53d3-b0f8-4024-a319-1703ede11193
13282024-09-01T21:00:15.052ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_e1c0d41c-ad07-4854-9798-d69fb32da3c7") failed: Property not found. retry in 1.20156251s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_e1c0d41c-ad07-4854-9798-d69fb32da3c7
13292024-09-01T21:00:15.106ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e6ddac78-e9f2-47ae-8121-95ee98f70ed8") failed: Property not found. retry in 530.141988ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e6ddac78-e9f2-47ae-8121-95ee98f70ed8
13302024-09-01T21:00:15.142ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_f9b91117-5154-4cc0-9de9-2e9efaaad0e6") failed: Property not found. retry in 571.679409ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_f9b91117-5154-4cc0-9de9-2e9efaaad0e6
13312024-09-01T21:00:15.380ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4091dac0-a803-4f41-a822-732fdb74b5db") failed: Property not found. retry in 675.757687ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4091dac0-a803-4f41-a822-732fdb74b5db
13322024-09-01T21:00:15.481ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0da3314b-38b4-4938-8831-8ec6d7490158") failed: Property not found. retry in 832.148568ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0da3314b-38b4-4938-8831-8ec6d7490158
13332024-09-01T21:00:15.614ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d0653118-7c84-43ff-9e70-8fe4cb6a2a64") failed: Property not found. retry in 1.268091695s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d0653118-7c84-43ff-9e70-8fe4cb6a2a64
13342024-09-01T21:00:15.633ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a0ecfdff-5e02-4b4f-863f-6f55a5749ea0") failed: Property not found. retry in 1.42448784s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a0ecfdff-5e02-4b4f-863f-6f55a5749ea0
13352024-09-01T21:00:15.670ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e6ddac78-e9f2-47ae-8121-95ee98f70ed8") failed: Property not found. retry in 821.437798ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e6ddac78-e9f2-47ae-8121-95ee98f70ed8
13362024-09-01T21:00:15.895ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_f9b91117-5154-4cc0-9de9-2e9efaaad0e6") failed: Property not found. retry in 1.200019107s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_f9b91117-5154-4cc0-9de9-2e9efaaad0e6
13372024-09-01T21:00:15.962ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_fcd4de7a-43df-4ccf-a747-0c745f76751c") failed: Property not found. retry in 1.08641941s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_fcd4de7a-43df-4ccf-a747-0c745f76751c
13382024-09-01T21:00:16.179ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_03bcf173-e5c2-4545-ac63-b6f41f5cbdda") failed: Property not found. retry in 1.345737912s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_03bcf173-e5c2-4545-ac63-b6f41f5cbdda
13392024-09-01T21:00:16.285ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_e1c0d41c-ad07-4854-9798-d69fb32da3c7") failed: Property not found. retry in 1.035255547s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_e1c0d41c-ad07-4854-9798-d69fb32da3c7
13402024-09-01T21:00:16.339ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0da3314b-38b4-4938-8831-8ec6d7490158") failed: Property not found. retry in 557.513792ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0da3314b-38b4-4938-8831-8ec6d7490158
13412024-09-01T21:00:16.515ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e6ddac78-e9f2-47ae-8121-95ee98f70ed8") failed: Property not found. retry in 1.048694935s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e6ddac78-e9f2-47ae-8121-95ee98f70ed8
13422024-09-01T21:00:16.905ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d0653118-7c84-43ff-9e70-8fe4cb6a2a64") failed: Property not found. retry in 1.216188465s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d0653118-7c84-43ff-9e70-8fe4cb6a2a64
13432024-09-01T21:00:16.926ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0da3314b-38b4-4938-8831-8ec6d7490158") failed: Property not found. retry in 630.446492ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0da3314b-38b4-4938-8831-8ec6d7490158
13442024-09-01T21:00:17.075ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_fcd4de7a-43df-4ccf-a747-0c745f76751c") failed: Property not found. retry in 993.99397ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_fcd4de7a-43df-4ccf-a747-0c745f76751c
13452024-09-01T21:00:17.114ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_f9b91117-5154-4cc0-9de9-2e9efaaad0e6") failed: Property not found. retry in 635.204087ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_f9b91117-5154-4cc0-9de9-2e9efaaad0e6
13462024-09-01T21:00:17.337ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_e1c0d41c-ad07-4854-9798-d69fb32da3c7") failed: Property not found. retry in 537.845783ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_e1c0d41c-ad07-4854-9798-d69fb32da3c7
13472024-09-01T21:00:17.542ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_03bcf173-e5c2-4545-ac63-b6f41f5cbdda") failed: Property not found. retry in 1.098780343s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_03bcf173-e5c2-4545-ac63-b6f41f5cbdda
13482024-09-01T21:00:17.576ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0da3314b-38b4-4938-8831-8ec6d7490158") failed: Property not found. retry in 812.620245ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0da3314b-38b4-4938-8831-8ec6d7490158
13492024-09-01T21:00:17.594ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e6ddac78-e9f2-47ae-8121-95ee98f70ed8") failed: Property not found. retry in 1.254042218s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e6ddac78-e9f2-47ae-8121-95ee98f70ed8
13502024-09-01T21:00:17.766ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_f9b91117-5154-4cc0-9de9-2e9efaaad0e6") failed: Property not found. retry in 1.32909209s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_f9b91117-5154-4cc0-9de9-2e9efaaad0e6
13512024-09-01T21:00:17.893ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_e1c0d41c-ad07-4854-9798-d69fb32da3c7") failed: Property not found. retry in 1.492569999s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_e1c0d41c-ad07-4854-9798-d69fb32da3c7
13522024-09-01T21:00:18.088ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_fcd4de7a-43df-4ccf-a747-0c745f76751c") failed: Property not found. retry in 1.005673253s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_fcd4de7a-43df-4ccf-a747-0c745f76751c
13532024-09-01T21:00:18.139ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d0653118-7c84-43ff-9e70-8fe4cb6a2a64") failed: Property not found. retry in 540.070831ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d0653118-7c84-43ff-9e70-8fe4cb6a2a64
13542024-09-01T21:00:18.406ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0da3314b-38b4-4938-8831-8ec6d7490158") failed: Property not found. retry in 1.406946674s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0da3314b-38b4-4938-8831-8ec6d7490158
13552024-09-01T21:00:18.657ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_03bcf173-e5c2-4545-ac63-b6f41f5cbdda") failed: Property not found. retry in 1.218789101s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_03bcf173-e5c2-4545-ac63-b6f41f5cbdda
13562024-09-01T21:00:18.695ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d0653118-7c84-43ff-9e70-8fe4cb6a2a64") failed: Property not found. retry in 628.83658ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d0653118-7c84-43ff-9e70-8fe4cb6a2a64
13572024-09-01T21:00:18.865ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e6ddac78-e9f2-47ae-8121-95ee98f70ed8") failed: Property not found. retry in 622.62272ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e6ddac78-e9f2-47ae-8121-95ee98f70ed8
13582024-09-01T21:00:19.108ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_fcd4de7a-43df-4ccf-a747-0c745f76751c") failed: Property not found. retry in 1.409660306s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_fcd4de7a-43df-4ccf-a747-0c745f76751c
13592024-09-01T21:00:19.119ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_f9b91117-5154-4cc0-9de9-2e9efaaad0e6") failed: Property not found. retry in 1.066960628s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_f9b91117-5154-4cc0-9de9-2e9efaaad0e6
13602024-09-01T21:00:19.339ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_d0653118-7c84-43ff-9e70-8fe4cb6a2a64") failed: Property not found. retry in 839.714785ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_d0653118-7c84-43ff-9e70-8fe4cb6a2a64
13612024-09-01T21:00:19.888ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_03bcf173-e5c2-4545-ac63-b6f41f5cbdda") failed: Property not found. retry in 1.474922509s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_03bcf173-e5c2-4545-ac63-b6f41f5cbdda
13622024-09-01T21:00:21.376ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_03bcf173-e5c2-4545-ac63-b6f41f5cbdda") failed: Property not found. retry in 1.415326663s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_03bcf173-e5c2-4545-ac63-b6f41f5cbdda
13632024-09-01T21:00:22.826ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_oximeter_7d3d53d3-b0f8-4024-a319-1703ede11193
13642024-09-01T21:00:22.827ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_crucible_pantry_95103d07-7ccb-4d09-b928-56c0fb8e4bb3
13652024-09-01T21:00:22.827ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_crucible_a7186ed2-a4e0-4db3-9300-6b0a04c483af
13662024-09-01T21:00:22.827ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_nexus_e6ddac78-e9f2-47ae-8121-95ee98f70ed8
13672024-09-01T21:00:22.827ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_nexus_f9b91117-5154-4cc0-9de9-2e9efaaad0e6
13682024-09-01T21:00:22.827ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_crucible_pantry_34c7db96-4f54-435b-bea8-b34ac92b151b
13692024-09-01T21:00:22.827ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_external_dns_e1c0d41c-ad07-4854-9798-d69fb32da3c7
13702024-09-01T21:00:22.827ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_crucible_4091dac0-a803-4f41-a822-732fdb74b5db
13712024-09-01T21:00:22.827ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_external_dns_d0653118-7c84-43ff-9e70-8fe4cb6a2a64
13722024-09-01T21:00:22.827ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_crucible_cf7f9d18-6fc4-4ee0-b35d-57ed8769bf3d
13732024-09-01T21:00:22.827ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_crucible_a0ecfdff-5e02-4b4f-863f-6f55a5749ea0
13742024-09-01T21:00:22.827ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_crucible_854f46a4-dab9-4716-b39d-9f437db37063
13752024-09-01T21:00:22.827ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_nexus_fcd4de7a-43df-4ccf-a747-0c745f76751c
13762024-09-01T21:00:22.827ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_clickhouse_03bcf173-e5c2-4545-ac63-b6f41f5cbdda
13772024-09-01T21:00:22.827ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_crucible_0da3314b-38b4-4938-8831-8ec6d7490158
13782024-09-01T21:00:22.827ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_crucible_6fe8da63-5815-4e50-aa5f-56ee726aff73
13792024-09-01T21:00:22.827ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_crucible_pantry_e7ead4f1-17b4-4c7d-843c-7f2e488501a0
13802024-09-01T21:00:22.828ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_crucible_b4673d99-2ce4-485e-a28e-9c3588338668
13812024-09-01T21:00:22.828ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3250 zone = oxz_crucible_93d2e0a4-500d-430c-b7ae-5ba369a25e48
13822024-09-01T21:00:22.830ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
13832024-09-01T21:00:22.835ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/c13a4209-582e-438f-b076-515144773dc5/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
13842024-09-01T21:00:22.839ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 31223885 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:38923 req_id = 086b8af5-fa70-4edd-88bb-f83fe4a76596 response_code = 204 uri = /omicron-zones
13852024-09-01T21:00:22.850ZINFOSledAgent (RSS): Finished setting up services file = sled-agent/src/rack_setup/service.rs:1286
13862024-09-01T21:00:22.850ZINFOSledAgent (RSS): Writing ledger to /pool/int/fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e/config/.rss-plan-completed.marker.tmp file = common/src/ledger.rs:199
13872024-09-01T21:00:22.851ZINFOSledAgent (RSS): Writing ledger to /pool/int/c13a4209-582e-438f-b076-515144773dc5/config/.rss-plan-completed.marker.tmp file = common/src/ledger.rs:199
13882024-09-01T21:00:22.856ZINFOSledAgent (RSS): Handing off control to Nexus file = sled-agent/src/rack_setup/service.rs:687
13892024-09-01T21:00:22.858ZINFOSledAgent (RSS): Nexus address: [fd00:1122:3344:101::a]:12221 file = sled-agent/src/rack_setup/service.rs:700
13902024-09-01T21:00:22.912ZINFOSledAgent (RSS): rack_network_config: RackNetworkConfigV2 { bfd: [], bgp: [], infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [ PortConfigV2 { addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.2.174, width: 24, }, ), vlan_id: None, }, ], autoneg: false, bgp_peers: [], lldp: None, port: "qsfp0", routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), local_pref: None, nexthop: 10.151.2.100, vlan_id: None, }, ], switch: Switch0, uplink_port_fec: None, uplink_port_speed: Speed40G, }, ], rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, } file = sled-agent/src/rack_setup/service.rs:867
13912024-09-01T21:00:22.918ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/82c87802-7334-4dc0-9ec8-482884e1fd48/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13922024-09-01T21:00:22.984ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/82c87802-7334-4dc0-9ec8-482884e1fd48/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13932024-09-01T21:00:23.138ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/82c87802-7334-4dc0-9ec8-482884e1fd48/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13942024-09-01T21:00:23.318ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/82c87802-7334-4dc0-9ec8-482884e1fd48/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13952024-09-01T21:00:23.413ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/82c87802-7334-4dc0-9ec8-482884e1fd48/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13962024-09-01T21:00:23.573ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/82c87802-7334-4dc0-9ec8-482884e1fd48/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13972024-09-01T21:00:23.738ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/82c87802-7334-4dc0-9ec8-482884e1fd48/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13982024-09-01T21:00:24.002ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/82c87802-7334-4dc0-9ec8-482884e1fd48/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13992024-09-01T21:00:24.337ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/82c87802-7334-4dc0-9ec8-482884e1fd48/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
14002024-09-01T21:00:24.912ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/82c87802-7334-4dc0-9ec8-482884e1fd48/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
14012024-09-01T21:00:25.455ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/82c87802-7334-4dc0-9ec8-482884e1fd48/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
14022024-09-01T21:00:26.351ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/82c87802-7334-4dc0-9ec8-482884e1fd48/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
14032024-09-01T21:00:26.752ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/82c87802-7334-4dc0-9ec8-482884e1fd48/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
14042024-09-01T21:00:27.656ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/82c87802-7334-4dc0-9ec8-482884e1fd48/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
14052024-09-01T21:00:29.101ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/82c87802-7334-4dc0-9ec8-482884e1fd48/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
14062024-09-01T21:00:30.690ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/82c87802-7334-4dc0-9ec8-482884e1fd48/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
14072024-09-01T21:00:32.351ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/82c87802-7334-4dc0-9ec8-482884e1fd48/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
14082024-09-01T21:00:33.223ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/d40a3584-3bdb-438a-9e24-58fb6b212748): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/nexus.rs:571 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
14092024-09-01T21:00:33.224ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/d40a3584-3bdb-438a-9e24-58fb6b212748): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/probe_manager.rs:230 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
14102024-09-01T21:00:33.423ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/82c87802-7334-4dc0-9ec8-482884e1fd48/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
14112024-09-01T21:00:33.661ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/d40a3584-3bdb-438a-9e24-58fb6b212748): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/nexus.rs:571 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
14122024-09-01T21:00:34.228ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/d40a3584-3bdb-438a-9e24-58fb6b212748): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/probe_manager.rs:230 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
14132024-09-01T21:00:34.973ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/82c87802-7334-4dc0-9ec8-482884e1fd48/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
14142024-09-01T21:00:35.232ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/d40a3584-3bdb-438a-9e24-58fb6b212748): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/probe_manager.rs:230 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
14152024-09-01T21:00:35.593ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/d40a3584-3bdb-438a-9e24-58fb6b212748): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/nexus.rs:571 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
14162024-09-01T21:00:36.237ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/d40a3584-3bdb-438a-9e24-58fb6b212748): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/probe_manager.rs:230 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
14172024-09-01T21:00:37.241ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/d40a3584-3bdb-438a-9e24-58fb6b212748): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/probe_manager.rs:230 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
14182024-09-01T21:00:37.594ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/d40a3584-3bdb-438a-9e24-58fb6b212748): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/nexus.rs:571 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
14192024-09-01T21:00:37.836ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/82c87802-7334-4dc0-9ec8-482884e1fd48/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
14202024-09-01T21:00:38.245ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/d40a3584-3bdb-438a-9e24-58fb6b212748): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/probe_manager.rs:230 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
14212024-09-01T21:00:41.148ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "e80f9918-a8cc-446c-a93c-1ff791d88105", "content-length": "173", "date": "Sun, 01 Sep 2024 21:00:40 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: rack with id \"82c87802-7334-4dc0-9ec8-482884e1fd48\"", request_id: "e80f9918-a8cc-446c-a93c-1ff791d88105" } file = sled-agent/src/rack_setup/service.rs:929
14222024-09-01T21:00:41.549ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:38567
14232024-09-01T21:00:41.552ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 75 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:38567 req_id = b39411e8-50bf-485c-8abd-82a4b4e33b8b response_code = 200 uri = /baseboard
14242024-09-01T21:00:41.726ZINFOSledAgent (NexusNotifierTask): Successfully put SledAgentInfo to nexus file = sled-agent/src/nexus.rs:544 new_generation = 1 old_generation = 1 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
14252024-09-01T21:00:42.644ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:53145
14262024-09-01T21:00:42.644ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 507 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:53145 req_id = 444ea3e2-bb6b-47ac-9c31-f356a85159e8 response_code = 200 uri = /vpc-routes
14272024-09-01T21:00:42.701ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:42854
14282024-09-01T21:00:42.705ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
14292024-09-01T21:00:42.813ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 111467 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:42854 req_id = 6257277c-576a-43c6-9a0d-92a9782f6c15 response_code = 200 uri = /inventory
14302024-09-01T21:00:42.814ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
14312024-09-01T21:00:42.814ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e/config/omicron-zones.json file = common/src/ledger.rs:177
14322024-09-01T21:00:42.815ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/c13a4209-582e-438f-b076-515144773dc5/config/omicron-zones.json file = common/src/ledger.rs:177
14332024-09-01T21:00:42.815ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1922 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:42854 req_id = 58c897fe-2245-4529-9a77-76d62c3b6c2b response_code = 200 uri = /omicron-zones
14342024-09-01T21:00:42.819ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 5740 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:53145 req_id = 8dea8c9e-b9cb-439a-9876-91954da52f72 response_code = 204 uri = /vpc-routes
14352024-09-01T21:00:42.983ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:39240
14362024-09-01T21:00:42.984ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 339 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:39240 req_id = 1689160e-96b3-4810-acd3-2b06684c4d8a response_code = 200 uri = /v2p
14372024-09-01T21:00:43.455ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:34599
14382024-09-01T21:00:43.457ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1387 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:34599 req_id = e9beaa6b-86cc-446c-bbc2-205a057709d0 response_code = 200 uri = /vpc-routes
14392024-09-01T21:00:43.481ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:46242
14402024-09-01T21:00:43.481ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
14412024-09-01T21:00:43.595ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 113849 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:46242 req_id = 9e30d152-6fa6-4ceb-8f4f-5d9f50c2a473 response_code = 200 uri = /inventory
14422024-09-01T21:00:43.595ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:41606
14432024-09-01T21:00:43.596ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
14442024-09-01T21:00:43.596ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e/config/omicron-zones.json file = common/src/ledger.rs:177
14452024-09-01T21:00:43.597ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1013 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:34599 req_id = f1d02066-8c23-4b89-ab74-3ab1b43d90ec response_code = 204 uri = /vpc-routes
14462024-09-01T21:00:43.597ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/c13a4209-582e-438f-b076-515144773dc5/config/omicron-zones.json file = common/src/ledger.rs:177
14472024-09-01T21:00:43.597ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1106 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:41606 req_id = ac9428e1-060a-40c6-9c71-66e63a12f0d2 response_code = 200 uri = /vpc-routes
14482024-09-01T21:00:43.598ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2634 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:46242 req_id = 396049a1-77e6-4d25-a710-bd491d149bf9 response_code = 200 uri = /omicron-zones
14492024-09-01T21:00:43.612ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 138 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:41606 req_id = 803a884a-20b3-488b-b991-d4f08c79635a response_code = 204 uri = /vpc-routes
14502024-09-01T21:00:43.635ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:38728
14512024-09-01T21:00:43.635ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 131 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:38728 req_id = 4d2b6905-77b6-4ae4-ba65-64072744f5b9 response_code = 200 uri = /v2p
14522024-09-01T21:00:43.645ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:52068
14532024-09-01T21:00:43.646ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1107 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:52068 req_id = 8c050522-ab39-47f9-832b-70bf106ca035 response_code = 200 uri = /v2p
14542024-09-01T21:00:43.703ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:49191
14552024-09-01T21:00:43.704ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
14562024-09-01T21:00:43.800ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 97482 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:49191 req_id = e4dd9b1b-0b67-4bb5-b9e9-59abccd8c9d3 response_code = 200 uri = /inventory
14572024-09-01T21:00:43.801ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
14582024-09-01T21:00:43.801ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e/config/omicron-zones.json file = common/src/ledger.rs:177
14592024-09-01T21:00:43.801ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/c13a4209-582e-438f-b076-515144773dc5/config/omicron-zones.json file = common/src/ledger.rs:177
14602024-09-01T21:00:43.801ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 722 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:49191 req_id = 84a2a324-bd64-4a87-805c-cd88aa894989 response_code = 200 uri = /omicron-zones
14612024-09-01T21:00:44.235ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:59416
14622024-09-01T21:00:44.236ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
14632024-09-01T21:00:44.330ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 94417 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:59416 req_id = d0e88de2-81bf-4707-89b7-978304525017 response_code = 200 uri = /inventory
14642024-09-01T21:00:44.346ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
14652024-09-01T21:00:44.346ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e/config/omicron-zones.json file = common/src/ledger.rs:177
14662024-09-01T21:00:44.347ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/c13a4209-582e-438f-b076-515144773dc5/config/omicron-zones.json file = common/src/ledger.rs:177
14672024-09-01T21:00:44.347ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1054 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:59416 req_id = babf2fb0-0789-44e0-84db-177956d58657 response_code = 200 uri = /omicron-zones
14682024-09-01T21:00:49.051ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:49841
14692024-09-01T21:00:49.053ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 30b8ab41-7fd4-46c2-a3ab-d5188430f837, kind: Service { id: e1c0d41c-ad07-4854-9798-d69fb32da3c7 }, name: Name("external-dns-e1c0d41c-ad07-4854-9798-d69fb32da3c7"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 141, 53])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d1fabfad-8660-4a68-bb4b-5a632f9c053c, kind: Service { id: d0653118-7c84-43ff-9e70-8fe4cb6a2a64 }, name: Name("external-dns-d0653118-7c84-43ff-9e70-8fe4cb6a2a64"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 250, 139])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 53c07f4a-42c7-4c20-a686-54474f5c9e46, kind: Service { id: e6ddac78-e9f2-47ae-8121-95ee98f70ed8 }, name: Name("nexus-e6ddac78-e9f2-47ae-8121-95ee98f70ed8"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 144, 191])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bae4dcf4-d7eb-4337-81e0-130002ad9304, kind: Service { id: f9b91117-5154-4cc0-9de9-2e9efaaad0e6 }, name: Name("nexus-f9b91117-5154-4cc0-9de9-2e9efaaad0e6"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 194, 40])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d3ef2e45-8e9b-444a-a0ae-d65db7d825bb, kind: Service { id: fcd4de7a-43df-4ccf-a747-0c745f76751c }, name: Name("nexus-fcd4de7a-43df-4ccf-a747-0c745f76751c"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 204, 184])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
14702024-09-01T21:00:49.053ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
14712024-09-01T21:00:49.053ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
14722024-09-01T21:00:49.053ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
14732024-09-01T21:00:49.053ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
14742024-09-01T21:00:49.053ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
14752024-09-01T21:00:49.053ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
14762024-09-01T21:00:49.053ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1916 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:49841 req_id = 47731f3a-5b84-4247-8c75-3d599e01adab response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
14772024-09-01T21:00:49.054ZINFOSledAgent (RSS): Handoff to Nexus is complete file = sled-agent/src/rack_setup/service.rs:939
14782024-09-01T21:00:49.408ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:54095
14792024-09-01T21:00:49.408ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
14802024-09-01T21:00:49.490ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 82105 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:54095 req_id = c69ae5bb-582b-471f-9c46-b17af4dd2b73 response_code = 200 uri = /inventory
14812024-09-01T21:00:49.491ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:53158
14822024-09-01T21:00:49.491ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
14832024-09-01T21:00:49.491ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e/config/omicron-zones.json file = common/src/ledger.rs:177
14842024-09-01T21:00:49.491ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 30b8ab41-7fd4-46c2-a3ab-d5188430f837, kind: Service { id: e1c0d41c-ad07-4854-9798-d69fb32da3c7 }, name: Name("external-dns-e1c0d41c-ad07-4854-9798-d69fb32da3c7"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 141, 53])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d1fabfad-8660-4a68-bb4b-5a632f9c053c, kind: Service { id: d0653118-7c84-43ff-9e70-8fe4cb6a2a64 }, name: Name("external-dns-d0653118-7c84-43ff-9e70-8fe4cb6a2a64"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 250, 139])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 53c07f4a-42c7-4c20-a686-54474f5c9e46, kind: Service { id: e6ddac78-e9f2-47ae-8121-95ee98f70ed8 }, name: Name("nexus-e6ddac78-e9f2-47ae-8121-95ee98f70ed8"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 144, 191])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bae4dcf4-d7eb-4337-81e0-130002ad9304, kind: Service { id: f9b91117-5154-4cc0-9de9-2e9efaaad0e6 }, name: Name("nexus-f9b91117-5154-4cc0-9de9-2e9efaaad0e6"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 194, 40])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d3ef2e45-8e9b-444a-a0ae-d65db7d825bb, kind: Service { id: fcd4de7a-43df-4ccf-a747-0c745f76751c }, name: Name("nexus-fcd4de7a-43df-4ccf-a747-0c745f76751c"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 204, 184])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
14852024-09-01T21:00:49.491ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
14862024-09-01T21:00:49.491ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
14872024-09-01T21:00:49.491ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
14882024-09-01T21:00:49.491ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
14892024-09-01T21:00:49.491ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
14902024-09-01T21:00:49.491ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
14912024-09-01T21:00:49.491ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 569 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:53158 req_id = dd1c3f38-4b7f-47cf-ad43-d6984d94cb54 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
14922024-09-01T21:00:49.491ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/c13a4209-582e-438f-b076-515144773dc5/config/omicron-zones.json file = common/src/ledger.rs:177
14932024-09-01T21:00:49.492ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 912 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:54095 req_id = 7ae36bdf-1253-41e1-b8f8-e6b29a9d8609 response_code = 200 uri = /omicron-zones
14942024-09-01T21:00:49.715ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:59454
14952024-09-01T21:00:49.716ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 30b8ab41-7fd4-46c2-a3ab-d5188430f837, kind: Service { id: e1c0d41c-ad07-4854-9798-d69fb32da3c7 }, name: Name("external-dns-e1c0d41c-ad07-4854-9798-d69fb32da3c7"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 141, 53])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d1fabfad-8660-4a68-bb4b-5a632f9c053c, kind: Service { id: d0653118-7c84-43ff-9e70-8fe4cb6a2a64 }, name: Name("external-dns-d0653118-7c84-43ff-9e70-8fe4cb6a2a64"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 250, 139])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 53c07f4a-42c7-4c20-a686-54474f5c9e46, kind: Service { id: e6ddac78-e9f2-47ae-8121-95ee98f70ed8 }, name: Name("nexus-e6ddac78-e9f2-47ae-8121-95ee98f70ed8"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 144, 191])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bae4dcf4-d7eb-4337-81e0-130002ad9304, kind: Service { id: f9b91117-5154-4cc0-9de9-2e9efaaad0e6 }, name: Name("nexus-f9b91117-5154-4cc0-9de9-2e9efaaad0e6"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 194, 40])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d3ef2e45-8e9b-444a-a0ae-d65db7d825bb, kind: Service { id: fcd4de7a-43df-4ccf-a747-0c745f76751c }, name: Name("nexus-fcd4de7a-43df-4ccf-a747-0c745f76751c"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 204, 184])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
14962024-09-01T21:00:49.716ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
14972024-09-01T21:00:49.716ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
14982024-09-01T21:00:49.716ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
14992024-09-01T21:00:49.716ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
15002024-09-01T21:00:49.716ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
15012024-09-01T21:00:49.716ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
15022024-09-01T21:00:49.716ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 985 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:59454 req_id = ec678aa9-2fb8-4a76-afc8-e9ed7abcd710 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
15032024-09-01T21:00:50.177ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:56678
15042024-09-01T21:00:50.177ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 30b8ab41-7fd4-46c2-a3ab-d5188430f837, kind: Service { id: e1c0d41c-ad07-4854-9798-d69fb32da3c7 }, name: Name("external-dns-e1c0d41c-ad07-4854-9798-d69fb32da3c7"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 141, 53])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d1fabfad-8660-4a68-bb4b-5a632f9c053c, kind: Service { id: d0653118-7c84-43ff-9e70-8fe4cb6a2a64 }, name: Name("external-dns-d0653118-7c84-43ff-9e70-8fe4cb6a2a64"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 250, 139])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 53c07f4a-42c7-4c20-a686-54474f5c9e46, kind: Service { id: e6ddac78-e9f2-47ae-8121-95ee98f70ed8 }, name: Name("nexus-e6ddac78-e9f2-47ae-8121-95ee98f70ed8"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 144, 191])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bae4dcf4-d7eb-4337-81e0-130002ad9304, kind: Service { id: f9b91117-5154-4cc0-9de9-2e9efaaad0e6 }, name: Name("nexus-f9b91117-5154-4cc0-9de9-2e9efaaad0e6"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 194, 40])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d3ef2e45-8e9b-444a-a0ae-d65db7d825bb, kind: Service { id: fcd4de7a-43df-4ccf-a747-0c745f76751c }, name: Name("nexus-fcd4de7a-43df-4ccf-a747-0c745f76751c"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 204, 184])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
15052024-09-01T21:00:50.177ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
15062024-09-01T21:00:50.177ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
15072024-09-01T21:00:50.177ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
15082024-09-01T21:00:50.177ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
15092024-09-01T21:00:50.177ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
15102024-09-01T21:00:50.177ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
15112024-09-01T21:00:50.177ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 282 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:56678 req_id = 6aa36c78-f92b-4445-8837-01d3d54d0480 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
15122024-09-01T21:01:02.000ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:62839
15132024-09-01T21:01:02.000ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 109 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:62839 req_id = 090934ce-3239-4e37-8712-c58d8b14fa63 response_code = 200 uri = /vpc-routes
15142024-09-01T21:01:02.010ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 761 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:62839 req_id = 2eea2df6-d80b-4328-8981-881b24547856 response_code = 204 uri = /vpc-routes
15152024-09-01T21:01:11.590ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:63510
15162024-09-01T21:01:11.590ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 668 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:63510 req_id = 9f8bd980-9185-4cff-824e-1dce51ba04f5 response_code = 200 uri = /baseboard
15172024-09-01T21:01:12.617ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51044
15182024-09-01T21:01:12.618ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 805 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:51044 req_id = db55c85d-f61c-4818-9e67-3ea21f862c48 response_code = 200 uri = /vpc-routes
15192024-09-01T21:01:12.633ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 140 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:51044 req_id = 26cb4a76-4a1f-4ab3-9dc3-adb7b98f649b response_code = 204 uri = /vpc-routes
15202024-09-01T21:01:12.723ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:35123
15212024-09-01T21:01:12.724ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 906 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:35123 req_id = be3a6748-f06c-405f-bae8-040cc2585328 response_code = 200 uri = /v2p
15222024-09-01T21:01:12.885ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:54962
15232024-09-01T21:01:12.887ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
15242024-09-01T21:01:12.963ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
15252024-09-01T21:01:13.040ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 154924 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:54962 req_id = b095dd56-087a-43b2-91c9-1fa4c2724c49 response_code = 204 uri = /switch-ports
15262024-09-01T21:01:13.043ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2616 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:54962 req_id = 0aaac000-a102-4e7d-b3f3-137850523cbc response_code = 200 uri = /network-bootstore-config
15272024-09-01T21:01:13.112ZINFOSledAgent (bootstore): Attempting to update network config with generation: 2, current_generation: 1 file = bootstore/src/schemes/v0/peer.rs:564 peer_id = pc-sock-i86pc
15282024-09-01T21:01:13.112ZINFOSledAgent (bootstore): Writing ledger to /pool/int/fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
15292024-09-01T21:01:13.115ZINFOSledAgent (bootstore): Writing ledger to /pool/int/c13a4209-582e-438f-b076-515144773dc5/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
15302024-09-01T21:01:13.116ZINFOSledAgent (bootstore): Broadcasting network config with generation 2 file = bootstore/src/schemes/v0/peer.rs:648 peer_id = pc-sock-i86pc
15312024-09-01T21:01:13.116ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 5410 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:54962 req_id = d07e20e6-26e7-4ce8-be88-5a7b006de57b response_code = 204 uri = /network-bootstore-config
15322024-09-01T21:01:13.408ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:45064
15332024-09-01T21:01:13.408ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 89 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:45064 req_id = a68f488c-ddeb-4da8-b9d8-0b4c972da118 response_code = 200 uri = /vpc-routes
15342024-09-01T21:01:13.420ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 132 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:45064 req_id = e6d5800a-a103-410e-9511-ad8587298166 response_code = 204 uri = /vpc-routes
15352024-09-01T21:01:13.513ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:53146
15362024-09-01T21:01:13.513ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 137 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:53146 req_id = 7534c06a-4caa-48f4-935e-2e4e681a0b0f response_code = 200 uri = /v2p
15372024-09-01T21:01:13.526ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:41561
15382024-09-01T21:01:13.527ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 880 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:41561 req_id = e74d9ba1-5ec5-462b-bc29-15cb793afec3 response_code = 200 uri = /vpc-routes
15392024-09-01T21:01:13.541ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:41559
15402024-09-01T21:01:13.541ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 134 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:41559 req_id = 29d683a5-918e-4e29-a77c-4713f7d7f44e response_code = 200 uri = /v2p
15412024-09-01T21:01:13.543ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 181 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:41561 req_id = 512f56d0-b145-4ee2-916c-bcab1f69581b response_code = 204 uri = /vpc-routes
15422024-09-01T21:01:13.590ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:40135
15432024-09-01T21:01:13.591ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
15442024-09-01T21:01:13.629ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
15452024-09-01T21:01:13.692ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 101238 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:40135 req_id = 3e0719d1-32cd-452d-a96f-ac47469bdbcf response_code = 204 uri = /switch-ports
15462024-09-01T21:01:13.692ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:34145
15472024-09-01T21:01:13.692ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
15482024-09-01T21:01:13.696ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3689 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:40135 req_id = 20096af7-78e2-4377-bfeb-641fbe5561a5 response_code = 200 uri = /network-bootstore-config
15492024-09-01T21:01:13.733ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
15502024-09-01T21:01:13.792ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 100033 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:34145 req_id = 21326e65-5575-445f-9701-b5a39316e3d5 response_code = 204 uri = /switch-ports
15512024-09-01T21:01:13.793ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 153 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:34145 req_id = 1caa2f5e-6c76-4414-9f4d-5afb3560458e response_code = 200 uri = /network-bootstore-config
15522024-09-01T21:01:20.794ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
15532024-09-01T21:01:41.626ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:39335
15542024-09-01T21:01:41.627ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 641 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:39335 req_id = 937d404f-75b7-4a75-878b-117285371ddd response_code = 200 uri = /baseboard
15552024-09-01T21:01:42.692ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:45232
15562024-09-01T21:01:42.692ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 93 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:45232 req_id = 5219114d-0dc8-4ac8-ba1e-1bf0be9ed6ec response_code = 200 uri = /vpc-routes
15572024-09-01T21:01:42.705ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 131 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:45232 req_id = 41c31fb2-dc63-4b9f-8455-4abdf79e0886 response_code = 204 uri = /vpc-routes
15582024-09-01T21:01:42.819ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:37627
15592024-09-01T21:01:42.819ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 163 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:37627 req_id = 992fb1c2-4884-4ae0-bd0c-8807bcb2cb51 response_code = 200 uri = /v2p
15602024-09-01T21:01:42.899ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:44855
15612024-09-01T21:01:42.899ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
15622024-09-01T21:01:42.935ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
15632024-09-01T21:01:42.995ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 95606 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:44855 req_id = 356f0829-b151-4aac-b013-56db1fa3e42a response_code = 204 uri = /switch-ports
15642024-09-01T21:01:42.995ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 105 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:44855 req_id = f24df576-2706-4d3a-b57b-c470ae38611a response_code = 200 uri = /network-bootstore-config
15652024-09-01T21:01:43.421ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:49986
15662024-09-01T21:01:43.421ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 96 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:49986 req_id = e27434fb-6096-4185-95b5-0536ed6d33e9 response_code = 200 uri = /vpc-routes
15672024-09-01T21:01:43.438ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 189 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:49986 req_id = a5fe7440-d37c-4d2c-a48e-eeb90016fba6 response_code = 204 uri = /vpc-routes
15682024-09-01T21:01:43.556ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:44448
15692024-09-01T21:01:43.556ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 55 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:44448 req_id = 89792edc-1874-4d5c-96c8-51de54ea629c response_code = 200 uri = /vpc-routes
15702024-09-01T21:01:43.566ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 102 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:44448 req_id = d69e300e-dc9f-4893-8cd0-93f7ccfb9c0c response_code = 204 uri = /vpc-routes
15712024-09-01T21:01:43.609ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:54351
15722024-09-01T21:01:43.609ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 124 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:54351 req_id = e4deab85-f6a7-4ec7-b1ae-ca336e47df48 response_code = 200 uri = /v2p
15732024-09-01T21:01:43.772ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:63239
15742024-09-01T21:01:43.772ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
15752024-09-01T21:01:43.807ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
15762024-09-01T21:01:43.862ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 90149 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:63239 req_id = 79945603-7e31-4a91-a025-f1638c8176f5 response_code = 204 uri = /switch-ports
15772024-09-01T21:01:43.862ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:36668
15782024-09-01T21:01:43.862ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:56198
15792024-09-01T21:01:43.863ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 380 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:63239 req_id = 5da55abb-5589-41da-8953-0f7754c765b3 response_code = 200 uri = /network-bootstore-config
15802024-09-01T21:01:43.863ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
15812024-09-01T21:01:43.863ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 970 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:36668 req_id = 8a671e2d-ee37-4b8b-8b6e-06b6631a35f3 response_code = 200 uri = /v2p
15822024-09-01T21:01:43.897ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
15832024-09-01T21:01:43.950ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 87897 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:56198 req_id = 2501bf00-7d1d-4c3a-854f-c49ba3518da5 response_code = 204 uri = /switch-ports
15842024-09-01T21:01:43.951ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 162 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:56198 req_id = 5862f047-e0ac-403d-b48c-1baa1d953fcd response_code = 200 uri = /network-bootstore-config
15852024-09-01T21:01:47.738ZINFOSledAgent (dropshot): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:46126 remote_addr = [fd00:1122:3344:101::d]:42698 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
15862024-09-01T21:01:47.808ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 69627 local_addr = [fd00:1122:3344:101::1]:46126 method = GET remote_addr = [fd00:1122:3344:101::d]:42698 req_id = 491732ef-97de-422c-9017-abf28358b254 response_code = 200 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748 uri = /d40a3584-3bdb-438a-9e24-58fb6b212748
15872024-09-01T21:02:11.659ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:41225
15882024-09-01T21:02:11.660ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 657 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:41225 req_id = ebd67513-c6ec-4557-834d-cc83af86b751 response_code = 200 uri = /baseboard
15892024-09-01T21:02:12.476ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:35185
15902024-09-01T21:02:12.476ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 99 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:35185 req_id = caffaaa7-bc65-41b3-aafd-2480b170251a response_code = 200 uri = /vpc-routes
15912024-09-01T21:02:12.488ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 869 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:35185 req_id = 3e6f33d2-e577-481a-a134-5bea586fc0fe response_code = 204 uri = /vpc-routes
15922024-09-01T21:02:12.655ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:56206
15932024-09-01T21:02:12.655ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 160 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:56206 req_id = b7bb022e-dce1-4403-89a3-b4a81462d782 response_code = 200 uri = /v2p
15942024-09-01T21:02:12.690ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:46956
15952024-09-01T21:02:12.690ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
15962024-09-01T21:02:12.727ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
15972024-09-01T21:02:12.780ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 89542 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:46956 req_id = d6147d36-b66c-45b7-8bde-a75645100167 response_code = 204 uri = /switch-ports
15982024-09-01T21:02:12.780ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 116 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:46956 req_id = 93e5a1ea-20e4-454d-848a-a588c5f7eab7 response_code = 200 uri = /network-bootstore-config
15992024-09-01T21:02:13.353ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:57805
16002024-09-01T21:02:13.353ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 53 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:57805 req_id = c4ce6a7c-c815-4be1-ad66-715fc005295e response_code = 200 uri = /vpc-routes
16012024-09-01T21:02:13.414ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:36264
16022024-09-01T21:02:13.414ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 39 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:36264 req_id = 8e0872fe-dc51-4571-8017-2869b28dfb51 response_code = 200 uri = /vpc-routes
16032024-09-01T21:02:13.425ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 112 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:36264 req_id = c7feb854-4a9f-4718-afd2-7686c7e1be40 response_code = 204 uri = /vpc-routes
16042024-09-01T21:02:13.440ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:62127
16052024-09-01T21:02:13.441ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 987 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:62127 req_id = 5a780a2e-e6bd-4ead-801b-e8eb0d41064c response_code = 200 uri = /v2p
16062024-09-01T21:02:13.449ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 74 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:57805 req_id = 9980fc91-9722-4873-853e-9db1351d8b8e response_code = 204 uri = /vpc-routes
16072024-09-01T21:02:13.605ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:56138
16082024-09-01T21:02:13.605ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 154 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:56138 req_id = 37a2b947-8427-41f5-aafa-03f232a46092 response_code = 200 uri = /v2p
16092024-09-01T21:02:13.647ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:59958
16102024-09-01T21:02:13.647ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
16112024-09-01T21:02:13.686ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
16122024-09-01T21:02:13.740ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 93118 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:59958 req_id = 0cc3e790-3fe4-46a2-a047-b826b59dbaff response_code = 204 uri = /switch-ports
16132024-09-01T21:02:13.740ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:58468
16142024-09-01T21:02:13.740ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
16152024-09-01T21:02:13.772ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
16162024-09-01T21:02:13.826ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 85636 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:58468 req_id = 77d909bd-f67d-4ed2-abcd-ded8a0ea98d3 response_code = 204 uri = /switch-ports
16172024-09-01T21:02:13.826ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 101 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:59958 req_id = 9c7682a4-ad55-4227-ade6-8bee75282eec response_code = 200 uri = /network-bootstore-config
16182024-09-01T21:02:13.826ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 178 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:58468 req_id = dac18fc5-90c8-44ca-bba2-898d0c212bde response_code = 200 uri = /network-bootstore-config
16192024-09-01T21:02:17.746ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 7698 local_addr = [fd00:1122:3344:101::1]:46126 method = GET remote_addr = [fd00:1122:3344:101::d]:42698 req_id = 279c5018-7883-46be-80bc-f8c869fce31f response_code = 200 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748 uri = /d40a3584-3bdb-438a-9e24-58fb6b212748
16202024-09-01T21:02:41.697ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:54415
16212024-09-01T21:02:41.698ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 731 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:54415 req_id = 7d2d3e7c-568f-4e1d-b8b3-416e0a587b1b response_code = 200 uri = /baseboard
16222024-09-01T21:02:42.576ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:57496
16232024-09-01T21:02:42.576ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 184 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:57496 req_id = c4540026-66df-438e-8699-82032015e1b3 response_code = 200 uri = /v2p
16242024-09-01T21:02:42.747ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:45872
16252024-09-01T21:02:42.747ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 62 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:45872 req_id = 9208e2a0-034e-4fc5-aec5-c8ea49e48831 response_code = 200 uri = /vpc-routes
16262024-09-01T21:02:42.757ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 128 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:45872 req_id = 20220f49-9f0f-4266-b6e9-cbd14af6f6f2 response_code = 204 uri = /vpc-routes
16272024-09-01T21:02:42.909ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:56499
16282024-09-01T21:02:42.909ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
16292024-09-01T21:02:42.943ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
16302024-09-01T21:02:42.998ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 89175 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:56499 req_id = 87a4627f-17bc-424c-8d95-79221eb4058c response_code = 204 uri = /switch-ports
16312024-09-01T21:02:42.999ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 974 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:56499 req_id = 31df16c1-791e-4394-b787-59fe2cf47b35 response_code = 200 uri = /network-bootstore-config
16322024-09-01T21:02:43.501ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:41643
16332024-09-01T21:02:43.501ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 106 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:41643 req_id = 63c1f050-1fe9-487d-a281-510ec3d22322 response_code = 200 uri = /vpc-routes
16342024-09-01T21:02:43.539ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:46910
16352024-09-01T21:02:43.540ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 153 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:46910 req_id = 6b87ed7b-7a6a-4045-9a9c-f246e4785b95 response_code = 200 uri = /v2p
16362024-09-01T21:02:43.551ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1311 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:41643 req_id = b8148776-2914-4134-9773-abc5a52684f7 response_code = 204 uri = /vpc-routes
16372024-09-01T21:02:43.646ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:55208
16382024-09-01T21:02:43.646ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 47 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:55208 req_id = c8b39037-40f8-45a0-8b10-69efa3cbe02d response_code = 200 uri = /vpc-routes
16392024-09-01T21:02:43.658ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 111 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:55208 req_id = 74611a1c-fc11-45cf-8b58-c2f2e91261d1 response_code = 204 uri = /vpc-routes
16402024-09-01T21:02:43.723ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:46336
16412024-09-01T21:02:43.723ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
16422024-09-01T21:02:43.761ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
16432024-09-01T21:02:43.819ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 96654 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:46336 req_id = 091afa26-dfe3-4989-9958-5c3ddc9e34a5 response_code = 204 uri = /switch-ports
16442024-09-01T21:02:43.819ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:55480
16452024-09-01T21:02:43.820ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 115 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:55480 req_id = 34302147-76c6-4a40-9c47-5217864a3152 response_code = 200 uri = /v2p
16462024-09-01T21:02:43.821ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 905 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:46336 req_id = 6583ebf3-4cf3-492e-b4a6-95283cdabdd4 response_code = 200 uri = /network-bootstore-config
16472024-09-01T21:02:43.870ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:33220
16482024-09-01T21:02:43.870ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
16492024-09-01T21:02:43.909ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
16502024-09-01T21:02:43.963ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 92588 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:33220 req_id = d8d940f1-26ff-4c7b-80f8-bc3851225f49 response_code = 204 uri = /switch-ports
16512024-09-01T21:02:43.963ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 192 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:33220 req_id = 4454804e-7913-4d04-924e-d0be00e3916e response_code = 200 uri = /network-bootstore-config
16522024-09-01T21:02:47.745ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 6640 local_addr = [fd00:1122:3344:101::1]:46126 method = GET remote_addr = [fd00:1122:3344:101::d]:42698 req_id = f9f73759-a573-43d3-b4d8-c00087d9ff35 response_code = 200 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748 uri = /d40a3584-3bdb-438a-9e24-58fb6b212748
16532024-09-01T21:03:11.736ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:63543
16542024-09-01T21:03:11.736ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 73 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:63543 req_id = 61d4dea3-c330-41a0-9cd8-7761623b54a6 response_code = 200 uri = /baseboard
16552024-09-01T21:03:12.601ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:59410
16562024-09-01T21:03:12.602ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 82 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:59410 req_id = 3a17b1e6-03bb-4d2e-8d88-19bab385c734 response_code = 200 uri = /vpc-routes
16572024-09-01T21:03:12.611ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 136 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:59410 req_id = cb3d7d1d-b468-4efe-8333-d8a9b1aced00 response_code = 204 uri = /vpc-routes
16582024-09-01T21:03:12.667ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:35380
16592024-09-01T21:03:12.667ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 149 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:35380 req_id = 4cab3751-4218-4c1f-899f-10d0e0b4b6de response_code = 200 uri = /v2p
16602024-09-01T21:03:12.704ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:38144
16612024-09-01T21:03:12.704ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
16622024-09-01T21:03:12.742ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
16632024-09-01T21:03:12.799ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 95497 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:38144 req_id = 1ddeb6e0-b528-42e5-9c91-4d1caaf95f73 response_code = 204 uri = /switch-ports
16642024-09-01T21:03:12.800ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 142 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:38144 req_id = c1dbc760-34e6-4e94-8777-544df4d7dc71 response_code = 200 uri = /network-bootstore-config
16652024-09-01T21:03:13.366ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:54083
16662024-09-01T21:03:13.366ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 62 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:54083 req_id = b0e6dcff-2ed8-494a-8e87-2a0e7db45b5e response_code = 200 uri = /vpc-routes
16672024-09-01T21:03:13.381ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 126 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:54083 req_id = f58f70a4-4c84-4c21-9cd9-cf0dec355bfd response_code = 204 uri = /vpc-routes
16682024-09-01T21:03:13.462ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:45857
16692024-09-01T21:03:13.462ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 913 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:45857 req_id = 28a93113-f22c-4f64-9e3b-1624559c7e9b response_code = 200 uri = /v2p
16702024-09-01T21:03:13.486ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:55401
16712024-09-01T21:03:13.486ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 35 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:55401 req_id = 39730a9e-aa7a-4eb4-a36e-6849952fa421 response_code = 200 uri = /vpc-routes
16722024-09-01T21:03:13.494ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 104 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:55401 req_id = 7e20e30e-3d3a-472f-bbe6-4de02c26e601 response_code = 204 uri = /vpc-routes
16732024-09-01T21:03:13.533ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:50485
16742024-09-01T21:03:13.533ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
16752024-09-01T21:03:13.572ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
16762024-09-01T21:03:13.625ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 92280 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:50485 req_id = 5f59a203-186b-4ddb-af4d-ee4d6231010a response_code = 204 uri = /switch-ports
16772024-09-01T21:03:13.625ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:57334
16782024-09-01T21:03:13.626ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:55410
16792024-09-01T21:03:13.626ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
16802024-09-01T21:03:13.630ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 4250 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:55410 req_id = d1473ce6-05be-4769-a8cd-e89188e65b49 response_code = 200 uri = /v2p
16812024-09-01T21:03:13.630ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 4453 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:50485 req_id = 7c65218b-d11d-4c67-bb00-dcf4a9c1e823 response_code = 200 uri = /network-bootstore-config
16822024-09-01T21:03:13.662ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
16832024-09-01T21:03:13.718ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 92390 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:57334 req_id = 16e6ffdf-339a-4c39-897b-487caf50d60c response_code = 204 uri = /switch-ports
16842024-09-01T21:03:13.719ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 874 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:57334 req_id = 164eab8e-0397-4381-8079-95793f72451d response_code = 200 uri = /network-bootstore-config
16852024-09-01T21:03:17.745ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 6740 local_addr = [fd00:1122:3344:101::1]:46126 method = GET remote_addr = [fd00:1122:3344:101::d]:42698 req_id = aa3ff49b-9fc7-4ad6-8e80-92dc334f7e47 response_code = 200 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748 uri = /d40a3584-3bdb-438a-9e24-58fb6b212748
16862024-09-01T21:03:41.774ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:37962
16872024-09-01T21:03:41.774ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 71 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:37962 req_id = 4c119dac-1748-4239-9d84-f63c0a697096 response_code = 200 uri = /baseboard
16882024-09-01T21:03:42.671ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:35508
16892024-09-01T21:03:42.671ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 93 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:35508 req_id = ec28fbbc-4ae6-4d5a-a197-b6bcfaf9fc39 response_code = 200 uri = /vpc-routes
16902024-09-01T21:03:42.686ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 163 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:35508 req_id = b6917970-4a4a-40ce-806e-5caa444ba4c7 response_code = 204 uri = /vpc-routes
16912024-09-01T21:03:42.841ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:49131
16922024-09-01T21:03:42.841ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 153 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:49131 req_id = a6c16c09-0f3c-40b7-931a-e4117be83d08 response_code = 200 uri = /v2p
16932024-09-01T21:03:42.843ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:57392
16942024-09-01T21:03:42.843ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
16952024-09-01T21:03:42.881ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
16962024-09-01T21:03:42.936ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 92928 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:57392 req_id = a8eb7e27-454c-46b3-81ed-ef3828dfe738 response_code = 204 uri = /switch-ports
16972024-09-01T21:03:42.936ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 214 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:57392 req_id = 3ac8ce42-1584-495a-bba1-03da1efa6c23 response_code = 200 uri = /network-bootstore-config
16982024-09-01T21:03:43.591ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:52613
16992024-09-01T21:03:43.592ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 937 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:52613 req_id = abb52740-230b-4c24-8335-9ec1f079e094 response_code = 200 uri = /vpc-routes
17002024-09-01T21:03:43.604ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 130 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:52613 req_id = 35e6c468-7c7a-47c1-9c4b-9f019da004e1 response_code = 204 uri = /vpc-routes
17012024-09-01T21:03:43.624ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:33218
17022024-09-01T21:03:43.624ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 37 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:33218 req_id = 9ca0c9d0-78dc-4b52-bfe7-cbee6965e569 response_code = 200 uri = /vpc-routes
17032024-09-01T21:03:43.642ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 157 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:33218 req_id = a98688d5-0be4-403c-bf64-b05296e45f54 response_code = 204 uri = /vpc-routes
17042024-09-01T21:03:43.720ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:50475
17052024-09-01T21:03:43.721ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
17062024-09-01T21:03:43.763ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
17072024-09-01T21:03:43.827ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 106684 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:50475 req_id = 880d2c54-1452-483f-9757-b610e6187c64 response_code = 204 uri = /switch-ports
17082024-09-01T21:03:43.827ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:34385
17092024-09-01T21:03:43.827ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:48205
17102024-09-01T21:03:43.827ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:42209
17112024-09-01T21:03:43.828ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 243 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:34385 req_id = 80dba29d-eab2-4f2e-ad7b-8695fe2f4174 response_code = 200 uri = /v2p
17122024-09-01T21:03:43.828ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 309 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:50475 req_id = d7fe760a-23e8-44ac-8218-2fb2ae74c078 response_code = 200 uri = /network-bootstore-config
17132024-09-01T21:03:43.828ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
17142024-09-01T21:03:43.828ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 319 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:42209 req_id = 7f123548-0d48-48f7-a535-1ee8408e6d41 response_code = 200 uri = /v2p
17152024-09-01T21:03:43.868ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
17162024-09-01T21:03:43.921ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 93669 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:48205 req_id = 09a0ecfb-efb8-4a69-8ac8-abec0ac40037 response_code = 204 uri = /switch-ports
17172024-09-01T21:03:43.922ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 160 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:48205 req_id = c1d20417-e1e3-4329-9da9-9ab076aaf25f response_code = 200 uri = /network-bootstore-config
17182024-09-01T21:03:47.764ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 12885 local_addr = [fd00:1122:3344:101::1]:46126 method = GET remote_addr = [fd00:1122:3344:101::d]:42698 req_id = 248f68eb-53c7-40dc-8a6c-7a285c5f7c33 response_code = 200 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748 uri = /d40a3584-3bdb-438a-9e24-58fb6b212748
17192024-09-01T21:04:11.812ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:49938
17202024-09-01T21:04:11.813ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 91 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:49938 req_id = 6bc705dc-7b81-46d8-b6a2-f32c2d737330 response_code = 200 uri = /baseboard
17212024-09-01T21:04:12.467ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:43604
17222024-09-01T21:04:12.468ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 868 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:43604 req_id = 7d58b21d-e7ab-4d00-812c-f8d8a10d7fcd response_code = 200 uri = /vpc-routes
17232024-09-01T21:04:12.487ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 154 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:43604 req_id = f4f00b91-67fa-47f4-90a7-6dc8a56e4a6d response_code = 204 uri = /vpc-routes
17242024-09-01T21:04:12.622ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:36696
17252024-09-01T21:04:12.623ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 140 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:36696 req_id = ac427640-488f-495c-994b-2967da7a7f85 response_code = 200 uri = /v2p
17262024-09-01T21:04:12.633ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:42664
17272024-09-01T21:04:12.633ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
17282024-09-01T21:04:12.671ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
17292024-09-01T21:04:12.724ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 90734 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:42664 req_id = bc51873e-36b2-48ec-8877-d59bc42b80e3 response_code = 204 uri = /switch-ports
17302024-09-01T21:04:12.725ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 92 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:42664 req_id = cacfa6f7-bd32-4b66-b697-510c0fb55d94 response_code = 200 uri = /network-bootstore-config
17312024-09-01T21:04:13.380ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:59940
17322024-09-01T21:04:13.380ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 60 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:59940 req_id = 47d9874c-96eb-49bc-b2ec-344e8880ece5 response_code = 200 uri = /vpc-routes
17332024-09-01T21:04:13.392ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 175 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:59940 req_id = 32796f46-fb44-4dd4-ae52-b72b727973e0 response_code = 204 uri = /vpc-routes
17342024-09-01T21:04:13.428ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:38597
17352024-09-01T21:04:13.428ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 56 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:38597 req_id = ae599d7b-f13b-4c43-9a1f-c3d026a34f82 response_code = 200 uri = /vpc-routes
17362024-09-01T21:04:13.446ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 115 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:38597 req_id = 17ef136c-aa5d-4dd5-96af-36862294ff19 response_code = 204 uri = /vpc-routes
17372024-09-01T21:04:13.491ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:63267
17382024-09-01T21:04:13.491ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
17392024-09-01T21:04:13.536ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
17402024-09-01T21:04:13.593ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 102846 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:63267 req_id = ed977af2-5778-4541-a8da-91d5930ddf81 response_code = 204 uri = /switch-ports
17412024-09-01T21:04:13.594ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:55135
17422024-09-01T21:04:13.594ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:60985
17432024-09-01T21:04:13.594ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 204 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:63267 req_id = 6f17d72f-b4dd-4a30-950d-895ebf037591 response_code = 200 uri = /network-bootstore-config
17442024-09-01T21:04:13.594ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 290 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:55135 req_id = 51b25535-d596-4bfc-bc76-00a87b56a740 response_code = 200 uri = /v2p
17452024-09-01T21:04:13.595ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
17462024-09-01T21:04:13.635ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
17472024-09-01T21:04:13.651ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:35823
17482024-09-01T21:04:13.652ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 295 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:35823 req_id = 6ccb8ebc-b83d-40ed-8bac-d1288abe17e4 response_code = 200 uri = /v2p
17492024-09-01T21:04:13.693ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 99493 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:60985 req_id = ca89b64e-2d14-4520-bdf0-6e519a7328cf response_code = 204 uri = /switch-ports
17502024-09-01T21:04:13.694ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 145 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:60985 req_id = 2cfa43cd-ce68-4dd4-baf6-c5e2c30668e5 response_code = 200 uri = /network-bootstore-config
17512024-09-01T21:04:17.830ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8367 local_addr = [fd00:1122:3344:101::1]:46126 method = GET remote_addr = [fd00:1122:3344:101::d]:42698 req_id = f2edd951-429a-4c3f-afe0-9ff642bd24bf response_code = 200 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748 uri = /d40a3584-3bdb-438a-9e24-58fb6b212748
17522024-09-01T21:04:41.850ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:61238
17532024-09-01T21:04:41.850ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 75 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:61238 req_id = 79a4952c-898a-4aa6-ade5-9b3a2eb39f0d response_code = 200 uri = /baseboard
17542024-09-01T21:04:42.650ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:64853
17552024-09-01T21:04:42.650ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 102 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:64853 req_id = 249715af-972e-4e83-9711-7382914efeff response_code = 200 uri = /vpc-routes
17562024-09-01T21:04:42.659ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 161 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:64853 req_id = 3b8fba33-90fc-48a9-9915-aff2cc933a74 response_code = 204 uri = /vpc-routes
17572024-09-01T21:04:43.008ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:33486
17582024-09-01T21:04:43.008ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 137 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:33486 req_id = d055dced-c9ee-40a4-9f38-1c7be0c94669 response_code = 200 uri = /v2p
17592024-09-01T21:04:43.039ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:38083
17602024-09-01T21:04:43.039ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
17612024-09-01T21:04:43.075ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
17622024-09-01T21:04:43.130ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 91324 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:38083 req_id = 8e239ad5-35ad-46d9-9a74-5e3169e1b037 response_code = 204 uri = /switch-ports
17632024-09-01T21:04:43.130ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 113 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:38083 req_id = 1b71b832-220b-4fda-9f93-b1bc6f7d303a response_code = 200 uri = /network-bootstore-config
17642024-09-01T21:04:43.492ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:56092
17652024-09-01T21:04:43.493ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 119 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:56092 req_id = 35316af5-f763-4b82-907e-61dbf82453b1 response_code = 200 uri = /vpc-routes
17662024-09-01T21:04:43.511ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:48637
17672024-09-01T21:04:43.511ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 100 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:48637 req_id = 67b10537-4b2b-42c9-8817-dd2fc1ee8b39 response_code = 200 uri = /vpc-routes
17682024-09-01T21:04:43.516ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1414 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:56092 req_id = e495d2f2-6401-4139-829a-7edac19b2b88 response_code = 204 uri = /vpc-routes
17692024-09-01T21:04:43.548ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 176 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:48637 req_id = bd51da43-cc4c-462d-abb7-46907006d903 response_code = 204 uri = /vpc-routes
17702024-09-01T21:04:43.728ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:62410
17712024-09-01T21:04:43.729ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 849 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:62410 req_id = 6b2b2da9-7b46-4c17-a58a-8b626676da72 response_code = 200 uri = /v2p
17722024-09-01T21:04:43.771ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:35401
17732024-09-01T21:04:43.771ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
17742024-09-01T21:04:43.810ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
17752024-09-01T21:04:43.871ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 99622 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:35401 req_id = 7d749f12-ead4-4ca2-9443-3de9293a6f31 response_code = 204 uri = /switch-ports
17762024-09-01T21:04:43.871ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:44543
17772024-09-01T21:04:43.871ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 254 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:44543 req_id = 11ef32d8-8711-4ceb-8400-9e6ffa21cadd response_code = 200 uri = /v2p
17782024-09-01T21:04:43.871ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 242 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:35401 req_id = ef42ad59-82a2-4b39-988f-9c7cdda5eb56 response_code = 200 uri = /network-bootstore-config
17792024-09-01T21:04:44.012ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:58323
17802024-09-01T21:04:44.012ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
17812024-09-01T21:04:44.049ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
17822024-09-01T21:04:44.103ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 91088 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:58323 req_id = 900935a5-419d-4bd8-9476-21eb505c184b response_code = 204 uri = /switch-ports
17832024-09-01T21:04:44.104ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 249 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:58323 req_id = 09011836-6118-45fc-9c13-1ae4c2ab8772 response_code = 200 uri = /network-bootstore-config
17842024-09-01T21:04:47.829ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 6759 local_addr = [fd00:1122:3344:101::1]:46126 method = GET remote_addr = [fd00:1122:3344:101::d]:42698 req_id = 6ec9b573-ee48-4b04-989e-bcb5299d0478 response_code = 200 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748 uri = /d40a3584-3bdb-438a-9e24-58fb6b212748
17852024-09-01T21:05:11.887ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:44643
17862024-09-01T21:05:11.887ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 111 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:44643 req_id = 4cc88d18-c6c0-421e-a373-af7b187a469b response_code = 200 uri = /baseboard
17872024-09-01T21:05:12.560ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:44306
17882024-09-01T21:05:12.560ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 82 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:44306 req_id = fb2fdf22-717a-4ec7-9f0a-38cdc18b6bf1 response_code = 200 uri = /vpc-routes
17892024-09-01T21:05:12.570ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 129 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:44306 req_id = d4a0126d-05cb-4d39-b816-4d030009d6c8 response_code = 204 uri = /vpc-routes
17902024-09-01T21:05:12.605ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:45184
17912024-09-01T21:05:12.605ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 111 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:45184 req_id = d9f23628-4185-4219-9c83-91800255050d response_code = 200 uri = /v2p
17922024-09-01T21:05:12.703ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:47225
17932024-09-01T21:05:12.703ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
17942024-09-01T21:05:12.740ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
17952024-09-01T21:05:12.794ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 90273 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:47225 req_id = 5fd3b28a-2f2a-4a1b-b33c-e53feb874f39 response_code = 204 uri = /switch-ports
17962024-09-01T21:05:12.794ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 192 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:47225 req_id = 42293408-8735-49dd-abc6-014ac0ae54de response_code = 200 uri = /network-bootstore-config
17972024-09-01T21:05:13.421ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:41626
17982024-09-01T21:05:13.421ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 60 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:41626 req_id = 728bf556-3ca9-43bb-b5b9-7a864b2efcce response_code = 200 uri = /vpc-routes
17992024-09-01T21:05:13.432ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1043 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:41626 req_id = 998182ab-c175-44c0-b2d9-1b69746b3158 response_code = 204 uri = /vpc-routes
18002024-09-01T21:05:13.515ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:58070
18012024-09-01T21:05:13.515ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 134 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:58070 req_id = d21248e2-44ff-4ca1-9e96-4f8669cc29c5 response_code = 200 uri = /v2p
18022024-09-01T21:05:13.531ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:46483
18032024-09-01T21:05:13.531ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
18042024-09-01T21:05:13.572ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
18052024-09-01T21:05:13.628ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 97206 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:46483 req_id = d76ee3fb-0be8-434a-8476-1956fce2e12e response_code = 204 uri = /switch-ports
18062024-09-01T21:05:13.629ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:38273
18072024-09-01T21:05:13.629ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:38627
18082024-09-01T21:05:13.629ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 232 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:46483 req_id = aa9b1020-274e-445b-84b8-e500513713d9 response_code = 200 uri = /network-bootstore-config
18092024-09-01T21:05:13.629ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 159 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:38627 req_id = 4c09eb05-f580-4f93-8142-1e9b65b4fda4 response_code = 200 uri = /vpc-routes
18102024-09-01T21:05:13.629ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 266 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:38273 req_id = adbf059f-1349-4d9c-81b5-0bd4e9877a31 response_code = 200 uri = /v2p
18112024-09-01T21:05:13.639ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 141 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:38627 req_id = bef5a431-063f-4405-a7e6-c10bf032f734 response_code = 204 uri = /vpc-routes
18122024-09-01T21:05:13.727ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:36043
18132024-09-01T21:05:13.727ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
18142024-09-01T21:05:13.763ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
18152024-09-01T21:05:13.820ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 92232 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:36043 req_id = aecd4cf4-bc18-4fcc-803b-2be3de14839c response_code = 204 uri = /switch-ports
18162024-09-01T21:05:13.821ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 756 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:36043 req_id = d8b4acef-89b7-4db3-9862-49a761be2624 response_code = 200 uri = /network-bootstore-config
18172024-09-01T21:05:17.829ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 7465 local_addr = [fd00:1122:3344:101::1]:46126 method = GET remote_addr = [fd00:1122:3344:101::d]:42698 req_id = be919840-adae-4c77-950f-566c833c257d response_code = 200 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748 uri = /d40a3584-3bdb-438a-9e24-58fb6b212748
18182024-09-01T21:05:41.920ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:64482
18192024-09-01T21:05:41.920ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 92 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:64482 req_id = 2c260c23-c359-42ad-a138-0de44c3f076d response_code = 200 uri = /baseboard
18202024-09-01T21:05:42.637ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:50611
18212024-09-01T21:05:42.637ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 115 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:50611 req_id = 80501780-8c93-4711-a6e3-f2903bbdce61 response_code = 200 uri = /vpc-routes
18222024-09-01T21:05:42.651ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 163 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:50611 req_id = e5df25c2-5e7a-463a-b776-3ccf6f6f1925 response_code = 204 uri = /vpc-routes
18232024-09-01T21:05:42.964ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:44384
18242024-09-01T21:05:42.964ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 146 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:44384 req_id = 4b2264b3-ba69-42cf-b2a0-92ef3b2c8d17 response_code = 200 uri = /v2p
18252024-09-01T21:05:43.001ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:65403
18262024-09-01T21:05:43.002ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
18272024-09-01T21:05:43.041ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
18282024-09-01T21:05:43.097ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 95447 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:65403 req_id = f0ea009e-572c-4dd7-a8e3-8ce4dcdb38a5 response_code = 204 uri = /switch-ports
18292024-09-01T21:05:43.097ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 111 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:65403 req_id = 2da44b34-5893-488e-a1e1-67b90d61ca7f response_code = 200 uri = /network-bootstore-config
18302024-09-01T21:05:43.136ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:42640
18312024-09-01T21:05:43.136ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 30b8ab41-7fd4-46c2-a3ab-d5188430f837, kind: Service { id: e1c0d41c-ad07-4854-9798-d69fb32da3c7 }, name: Name("external-dns-e1c0d41c-ad07-4854-9798-d69fb32da3c7"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 141, 53])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d1fabfad-8660-4a68-bb4b-5a632f9c053c, kind: Service { id: d0653118-7c84-43ff-9e70-8fe4cb6a2a64 }, name: Name("external-dns-d0653118-7c84-43ff-9e70-8fe4cb6a2a64"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 250, 139])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 53c07f4a-42c7-4c20-a686-54474f5c9e46, kind: Service { id: e6ddac78-e9f2-47ae-8121-95ee98f70ed8 }, name: Name("nexus-e6ddac78-e9f2-47ae-8121-95ee98f70ed8"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 144, 191])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bae4dcf4-d7eb-4337-81e0-130002ad9304, kind: Service { id: f9b91117-5154-4cc0-9de9-2e9efaaad0e6 }, name: Name("nexus-f9b91117-5154-4cc0-9de9-2e9efaaad0e6"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 194, 40])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d3ef2e45-8e9b-444a-a0ae-d65db7d825bb, kind: Service { id: fcd4de7a-43df-4ccf-a747-0c745f76751c }, name: Name("nexus-fcd4de7a-43df-4ccf-a747-0c745f76751c"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 204, 184])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
18322024-09-01T21:05:43.136ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
18332024-09-01T21:05:43.136ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
18342024-09-01T21:05:43.136ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18352024-09-01T21:05:43.136ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18362024-09-01T21:05:43.136ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
18372024-09-01T21:05:43.136ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18382024-09-01T21:05:43.136ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 424 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:42640 req_id = d6606546-45d3-41b4-a13d-375420f9043b response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
18392024-09-01T21:05:43.465ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:38393
18402024-09-01T21:05:43.467ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1011 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:38393 req_id = 3c89cb3f-f497-4f39-8b2a-007517b5c5b1 response_code = 200 uri = /vpc-routes
18412024-09-01T21:05:43.481ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 186 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:38393 req_id = 1e572f45-9cce-4d57-9f91-b1d73f3093cf response_code = 204 uri = /vpc-routes
18422024-09-01T21:05:43.670ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:61617
18432024-09-01T21:05:43.670ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 57 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:61617 req_id = 6b0943d9-8b1d-4ac5-89e6-0e24e397915e response_code = 200 uri = /vpc-routes
18442024-09-01T21:05:43.682ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 102 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:61617 req_id = 0f4be651-1f34-4467-ab0b-9d8c93f1938f response_code = 204 uri = /vpc-routes
18452024-09-01T21:05:43.715ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:60901
18462024-09-01T21:05:43.715ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 119 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:60901 req_id = 2178a06c-58e9-4508-9d00-c330038af307 response_code = 200 uri = /v2p
18472024-09-01T21:05:43.807ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:37939
18482024-09-01T21:05:43.807ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
18492024-09-01T21:05:43.848ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
18502024-09-01T21:05:43.908ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 100761 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:37939 req_id = d2999e5e-6e27-4725-8ef0-76b23eb59f85 response_code = 204 uri = /switch-ports
18512024-09-01T21:05:43.908ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:40939
18522024-09-01T21:05:43.908ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:57785
18532024-09-01T21:05:43.908ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
18542024-09-01T21:05:43.908ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 298 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:37939 req_id = 07d9657d-1a9e-4211-bb47-89417779f9e3 response_code = 200 uri = /network-bootstore-config
18552024-09-01T21:05:43.912ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 4268 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:57785 req_id = e988fcd2-1712-40f5-979f-7b7102dc6ea6 response_code = 200 uri = /v2p
18562024-09-01T21:05:43.925ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:40593
18572024-09-01T21:05:43.925ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 30b8ab41-7fd4-46c2-a3ab-d5188430f837, kind: Service { id: e1c0d41c-ad07-4854-9798-d69fb32da3c7 }, name: Name("external-dns-e1c0d41c-ad07-4854-9798-d69fb32da3c7"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 141, 53])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d1fabfad-8660-4a68-bb4b-5a632f9c053c, kind: Service { id: d0653118-7c84-43ff-9e70-8fe4cb6a2a64 }, name: Name("external-dns-d0653118-7c84-43ff-9e70-8fe4cb6a2a64"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 250, 139])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 53c07f4a-42c7-4c20-a686-54474f5c9e46, kind: Service { id: e6ddac78-e9f2-47ae-8121-95ee98f70ed8 }, name: Name("nexus-e6ddac78-e9f2-47ae-8121-95ee98f70ed8"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 144, 191])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bae4dcf4-d7eb-4337-81e0-130002ad9304, kind: Service { id: f9b91117-5154-4cc0-9de9-2e9efaaad0e6 }, name: Name("nexus-f9b91117-5154-4cc0-9de9-2e9efaaad0e6"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 194, 40])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d3ef2e45-8e9b-444a-a0ae-d65db7d825bb, kind: Service { id: fcd4de7a-43df-4ccf-a747-0c745f76751c }, name: Name("nexus-fcd4de7a-43df-4ccf-a747-0c745f76751c"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 204, 184])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
18582024-09-01T21:05:43.926ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
18592024-09-01T21:05:43.926ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
18602024-09-01T21:05:43.926ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18612024-09-01T21:05:43.926ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18622024-09-01T21:05:43.926ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
18632024-09-01T21:05:43.926ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18642024-09-01T21:05:43.926ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 708 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:40593 req_id = 33d1fa9c-448b-48c9-a2c2-4791287813a5 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
18652024-09-01T21:05:43.944ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
18662024-09-01T21:05:43.952ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:61515
18672024-09-01T21:05:43.952ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 30b8ab41-7fd4-46c2-a3ab-d5188430f837, kind: Service { id: e1c0d41c-ad07-4854-9798-d69fb32da3c7 }, name: Name("external-dns-e1c0d41c-ad07-4854-9798-d69fb32da3c7"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 141, 53])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d1fabfad-8660-4a68-bb4b-5a632f9c053c, kind: Service { id: d0653118-7c84-43ff-9e70-8fe4cb6a2a64 }, name: Name("external-dns-d0653118-7c84-43ff-9e70-8fe4cb6a2a64"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 250, 139])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 53c07f4a-42c7-4c20-a686-54474f5c9e46, kind: Service { id: e6ddac78-e9f2-47ae-8121-95ee98f70ed8 }, name: Name("nexus-e6ddac78-e9f2-47ae-8121-95ee98f70ed8"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 144, 191])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bae4dcf4-d7eb-4337-81e0-130002ad9304, kind: Service { id: f9b91117-5154-4cc0-9de9-2e9efaaad0e6 }, name: Name("nexus-f9b91117-5154-4cc0-9de9-2e9efaaad0e6"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 194, 40])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d3ef2e45-8e9b-444a-a0ae-d65db7d825bb, kind: Service { id: fcd4de7a-43df-4ccf-a747-0c745f76751c }, name: Name("nexus-fcd4de7a-43df-4ccf-a747-0c745f76751c"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 204, 184])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
18682024-09-01T21:05:43.952ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
18692024-09-01T21:05:43.952ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
18702024-09-01T21:05:43.952ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18712024-09-01T21:05:43.952ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18722024-09-01T21:05:43.952ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
18732024-09-01T21:05:43.952ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18742024-09-01T21:05:43.952ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 431 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:61515 req_id = 9dc70357-e61a-4a2b-b2de-58f754588bad response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
18752024-09-01T21:05:43.995ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 87343 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:40939 req_id = d71ee1df-3d49-48cf-a690-57d6a895c8bf response_code = 204 uri = /switch-ports
18762024-09-01T21:05:43.996ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 152 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:40939 req_id = 8fc03b48-baf4-45e3-8312-4bc3183d07bc response_code = 200 uri = /network-bootstore-config
18772024-09-01T21:05:47.829ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 7331 local_addr = [fd00:1122:3344:101::1]:46126 method = GET remote_addr = [fd00:1122:3344:101::d]:42698 req_id = c864e3c0-97ef-4e53-bfa3-72dffb4de765 response_code = 200 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748 uri = /d40a3584-3bdb-438a-9e24-58fb6b212748
18782024-09-01T21:06:05.878ZINFOSledAgent (auto-cleanup-task): running automatic periodic zone bundle cleanup file = sled-agent/src/zone_bundle.rs:197
18792024-09-01T21:06:05.878ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
18802024-09-01T21:06:05.919ZINFOSledAgent (auto-cleanup-task): top of bundle cleanup loop file = sled-agent/src/zone_bundle.rs:186 next_cleanup = Instant { tv_sec: 4182, tv_nsec: 436393387 } time_to_next_cleanup = 599.9999954s
18812024-09-01T21:06:11.954ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:47694
18822024-09-01T21:06:11.954ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 652 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:47694 req_id = cc27dac2-9b4b-4dfb-ba07-4ab1f2b1e840 response_code = 200 uri = /baseboard
18832024-09-01T21:06:12.507ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:61465
18842024-09-01T21:06:12.507ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 108 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:61465 req_id = d3544d43-5c32-4afb-acd4-2401b3fae8ae response_code = 200 uri = /vpc-routes
18852024-09-01T21:06:12.517ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 147 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:61465 req_id = d8e89283-0f56-48a7-b0df-99b76a570b8b response_code = 204 uri = /vpc-routes
18862024-09-01T21:06:12.727ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:52984
18872024-09-01T21:06:12.727ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 160 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:52984 req_id = 6868043a-0016-492c-8e38-92a70694e777 response_code = 200 uri = /v2p
18882024-09-01T21:06:12.789ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:61262
18892024-09-01T21:06:12.789ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
18902024-09-01T21:06:12.826ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
18912024-09-01T21:06:12.882ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 92522 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:61262 req_id = e8e53080-b2d3-4955-93e0-75fa2e120873 response_code = 204 uri = /switch-ports
18922024-09-01T21:06:12.882ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 114 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:61262 req_id = ccd21e85-e7f4-4307-a254-d18883d4bb5e response_code = 200 uri = /network-bootstore-config
18932024-09-01T21:06:13.412ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:46556
18942024-09-01T21:06:13.413ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 999 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:46556 req_id = fdeeaf7d-630b-4fa5-8270-4f3f49c36772 response_code = 200 uri = /vpc-routes
18952024-09-01T21:06:13.418ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:57365
18962024-09-01T21:06:13.419ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 48 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:57365 req_id = d8df9ec1-6e4b-4fe3-96b5-4ff93914a97f response_code = 200 uri = /vpc-routes
18972024-09-01T21:06:13.428ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 131 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:57365 req_id = 36d8ba21-f2ab-42d0-ba77-b827313471ac response_code = 204 uri = /vpc-routes
18982024-09-01T21:06:13.491ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:33197
18992024-09-01T21:06:13.492ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 146 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:33197 req_id = 0a62b609-46b1-4e06-819c-e660c9a7755d response_code = 200 uri = /v2p
19002024-09-01T21:06:13.494ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 67 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:46556 req_id = bc53f309-635f-43db-96da-59209640e6ba response_code = 204 uri = /vpc-routes
19012024-09-01T21:06:13.556ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:62850
19022024-09-01T21:06:13.557ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 711 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:62850 req_id = 6a834344-86f2-431f-be4d-ec75637919c2 response_code = 200 uri = /v2p
19032024-09-01T21:06:13.558ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:38542
19042024-09-01T21:06:13.558ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
19052024-09-01T21:06:13.597ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
19062024-09-01T21:06:13.650ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 91553 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:38542 req_id = 753e856f-d11b-4c9c-abca-5d425be53015 response_code = 204 uri = /switch-ports
19072024-09-01T21:06:13.650ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:36967
19082024-09-01T21:06:13.651ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
19092024-09-01T21:06:13.651ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 282 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:38542 req_id = 034ae498-fc41-4d78-9fb1-826fa1252848 response_code = 200 uri = /network-bootstore-config
19102024-09-01T21:06:13.686ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
19112024-09-01T21:06:13.740ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 89863 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:36967 req_id = 890faabd-ba13-46a7-a6f4-e9bd3081964b response_code = 204 uri = /switch-ports
19122024-09-01T21:06:13.741ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 147 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:36967 req_id = b7e8589e-74d2-4262-a30c-e5cc04a13967 response_code = 200 uri = /network-bootstore-config
19132024-09-01T21:06:17.829ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 6450 local_addr = [fd00:1122:3344:101::1]:46126 method = GET remote_addr = [fd00:1122:3344:101::d]:42698 req_id = 27d1e0f1-508e-4999-a1ef-e1a44f719c3c response_code = 200 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748 uri = /d40a3584-3bdb-438a-9e24-58fb6b212748
19142024-09-01T21:06:20.796ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
19152024-09-01T21:06:41.988ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:32773
19162024-09-01T21:06:41.988ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 583 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:32773 req_id = e7632b2f-0a73-4639-a733-b85edc9e6704 response_code = 200 uri = /baseboard
19172024-09-01T21:06:42.609ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:65487
19182024-09-01T21:06:42.609ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 81 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:65487 req_id = 9a9790b3-c371-4521-b111-fe01e65e57ff response_code = 200 uri = /vpc-routes
19192024-09-01T21:06:42.620ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 814 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:65487 req_id = 810c39f1-9875-4570-a218-c3a8f51a6da5 response_code = 204 uri = /vpc-routes
19202024-09-01T21:06:42.697ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:37966
19212024-09-01T21:06:42.697ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 123 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:37966 req_id = 13c0c293-e78e-4c07-8316-5a74e4840777 response_code = 200 uri = /v2p
19222024-09-01T21:06:42.699ZINFOSledAgent (PortManager): Mapping virtual NIC to physical host file = illumos-utils/src/opte/port_manager.rs:778 mapping = VirtualNetworkInterfaceHost { virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 253, 94, 45])), physical_host_ip: fd00:1122:3344:101::1, vni: Vni(5878965) }
19232024-09-01T21:06:42.699ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1703 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:37966 req_id = 725550f1-644a-4ecf-a72e-6da12c01239d response_code = 204 uri = /v2p
19242024-09-01T21:06:42.755ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:47327
19252024-09-01T21:06:42.755ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
19262024-09-01T21:06:42.794ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
19272024-09-01T21:06:42.850ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 94968 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:47327 req_id = f52faea7-e32a-42eb-8cdc-69b9696c57a7 response_code = 204 uri = /switch-ports
19282024-09-01T21:06:42.851ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 118 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:47327 req_id = d00f7f44-b774-4640-b848-745c6550f2fc response_code = 200 uri = /network-bootstore-config
19292024-09-01T21:06:42.886ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:37429
19302024-09-01T21:06:42.887ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 821 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:37429 req_id = 708b2139-8741-4c7b-8a25-27c13b4333a5 response_code = 200 uri = /v2p
19312024-09-01T21:06:43.558ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:56952
19322024-09-01T21:06:43.567ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:48576
19332024-09-01T21:06:43.567ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 80 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:48576 req_id = 7d57d394-92f9-4b81-a4a7-8fd9bf87ed4e response_code = 200 uri = /vpc-routes
19342024-09-01T21:06:43.572ZINFOSledAgent (InstanceManager): ensuring instance is registered file = sled-agent/src/instance_manager.rs:570 hardware = InstanceHardware { properties: InstanceProperties { ncpus: InstanceCpuCount(2), memory: ByteCount(1073741824), hostname: Hostname("localshark") }, nics: [NetworkInterface { id: 6a7e3435-d183-46c2-83c3-9514837aa8a1, kind: Instance { id: e833a082-213e-41ac-a531-e595eca5375c }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 253, 94, 45])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(5878965), primary: true, slot: 0, transit_ips: [] }], source_nat: SourceNatConfig { ip: 10.151.2.175, first_port: 0, last_port: 16383 }, ephemeral_ip: Some(10.151.2.176), floating_ips: [], firewall_rules: [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 6a7e3435-d183-46c2-83c3-9514837aa8a1, kind: Instance { id: e833a082-213e-41ac-a531-e595eca5375c }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 253, 94, 45])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(5878965), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 6a7e3435-d183-46c2-83c3-9514837aa8a1, kind: Instance { id: e833a082-213e-41ac-a531-e595eca5375c }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 253, 94, 45])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(5878965), primary: true, slot: 0, transit_ips: [] }], filter_hosts: Some([Vpc(Vni(5878965))]), filter_ports: None, filter_protocols: None, action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 6a7e3435-d183-46c2-83c3-9514837aa8a1, kind: Instance { id: e833a082-213e-41ac-a531-e595eca5375c }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 253, 94, 45])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(5878965), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(22), last: L4Port(22) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }], dhcp_config: DhcpConfig { dns_servers: [1.1.1.1, 9.9.9.9], host_domain: None, search_domains: [] }, disks: [DiskRequest { device: "nvme", name: "disk-f2ec047645f", read_only: false, slot: Slot(0), volume_construction_request: Volume { block_size: 512, id: ffda79b0-17cb-4e26-97e3-d10abb3cb17e, read_only_parent: Some(Volume { block_size: 512, id: e7333b9b-6cdc-4d92-a0fb-c9ac4f899f70, read_only_parent: None, sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 2, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 05440cac-2490-470d-93eb-20b6ff77a16a, key: Some("dBG/f9JkoAf5e7PBN6LK6rHnIrAAshYwcjc7DJg/4aU="), key_pem: None, lossy: false, read_only: true, root_cert_pem: None, target: ["[fd00:1122:3344:101::13]:19002", "[fd00:1122:3344:101::15]:19001", "[fd00:1122:3344:101::1a]:19001"] } }] }), sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 1, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: ffda79b0-17cb-4e26-97e3-d10abb3cb17e, key: Some("0o1W0deCKDQUtZoMGGFx9iUeSoaYqxHFQ5AK6/eMNtk="), key_pem: None, lossy: false, read_only: false, root_cert_pem: None, target: ["[fd00:1122:3344:101::13]:19003", "[fd00:1122:3344:101::15]:19002", "[fd00:1122:3344:101::14]:19000"] } }] } }], cloud_init_bytes: Some(..) } instance_id = e833a082-213e-41ac-a531-e595eca5375c instance_runtime = InstanceRuntimeState { propolis_id: Some(93dbd91d-c521-444f-a889-3e26eca48571 (propolis)), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2024-09-01T21:06:42.384311820Z } metadata = InstanceMetadata { silo_id: 88894989-47c5-4507-b48e-fb7d04b750bc, project_id: 26a656fb-1640-48d0-bf7b-88b941d0219a } propolis_addr = [fd00:1122:3344:101::1:0]:12400 propolis_id = 93dbd91d-c521-444f-a889-3e26eca48571 vmm_runtime = VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2024-09-01T21:06:43.556344159Z }
19352024-09-01T21:06:43.572ZINFOSledAgent (InstanceManager): registering new instance file = sled-agent/src/instance_manager.rs:603 instance_id = e833a082-213e-41ac-a531-e595eca5375c propolis_id = 93dbd91d-c521-444f-a889-3e26eca48571
19362024-09-01T21:06:43.572ZINFOSledAgent (InstanceManager): initializing new Instance file = sled-agent/src/instance.rs:991 instance_id = e833a082-213e-41ac-a531-e595eca5375c migration_id = None propolis_id = 93dbd91d-c521-444f-a889-3e26eca48571 state = InstanceInitialState { hardware: InstanceHardware { properties: InstanceProperties { ncpus: InstanceCpuCount(2), memory: ByteCount(1073741824), hostname: Hostname("localshark") }, nics: [NetworkInterface { id: 6a7e3435-d183-46c2-83c3-9514837aa8a1, kind: Instance { id: e833a082-213e-41ac-a531-e595eca5375c }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 253, 94, 45])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(5878965), primary: true, slot: 0, transit_ips: [] }], source_nat: SourceNatConfig { ip: 10.151.2.175, first_port: 0, last_port: 16383 }, ephemeral_ip: Some(10.151.2.176), floating_ips: [], firewall_rules: [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 6a7e3435-d183-46c2-83c3-9514837aa8a1, kind: Instance { id: e833a082-213e-41ac-a531-e595eca5375c }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 253, 94, 45])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(5878965), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 6a7e3435-d183-46c2-83c3-9514837aa8a1, kind: Instance { id: e833a082-213e-41ac-a531-e595eca5375c }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 253, 94, 45])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(5878965), primary: true, slot: 0, transit_ips: [] }], filter_hosts: Some([Vpc(Vni(5878965))]), filter_ports: None, filter_protocols: None, action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 6a7e3435-d183-46c2-83c3-9514837aa8a1, kind: Instance { id: e833a082-213e-41ac-a531-e595eca5375c }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 253, 94, 45])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(5878965), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(22), last: L4Port(22) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }], dhcp_config: DhcpConfig { dns_servers: [1.1.1.1, 9.9.9.9], host_domain: None, search_domains: [] }, disks: [DiskRequest { device: "nvme", name: "disk-f2ec047645f", read_only: false, slot: Slot(0), volume_construction_request: Volume { block_size: 512, id: ffda79b0-17cb-4e26-97e3-d10abb3cb17e, read_only_parent: Some(Volume { block_size: 512, id: e7333b9b-6cdc-4d92-a0fb-c9ac4f899f70, read_only_parent: None, sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 2, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 05440cac-2490-470d-93eb-20b6ff77a16a, key: Some("dBG/f9JkoAf5e7PBN6LK6rHnIrAAshYwcjc7DJg/4aU="), key_pem: None, lossy: false, read_only: true, root_cert_pem: None, target: ["[fd00:1122:3344:101::13]:19002", "[fd00:1122:3344:101::15]:19001", "[fd00:1122:3344:101::1a]:19001"] } }] }), sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 1, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: ffda79b0-17cb-4e26-97e3-d10abb3cb17e, key: Some("0o1W0deCKDQUtZoMGGFx9iUeSoaYqxHFQ5AK6/eMNtk="), key_pem: None, lossy: false, read_only: false, root_cert_pem: None, target: ["[fd00:1122:3344:101::13]:19003", "[fd00:1122:3344:101::15]:19002", "[fd00:1122:3344:101::14]:19000"] } }] } }], cloud_init_bytes: Some(..) }, vmm_runtime: VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2024-09-01T21:06:43.556344159Z }, propolis_addr: [fd00:1122:3344:101::1:0]:12400, migration_id: None }
19372024-09-01T21:06:43.572ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 14346 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:56952 req_id = bc213add-59b2-4478-bf29-089018d5e3fc response_code = 200 uri = /vmms/93dbd91d-c521-444f-a889-3e26eca48571
19382024-09-01T21:06:43.585ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1006 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:48576 req_id = d3986313-4961-4e42-8413-f266a923f59e response_code = 204 uri = /vpc-routes
19392024-09-01T21:06:43.654ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:50901
19402024-09-01T21:06:43.655ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 717 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:50901 req_id = 915e4a50-83df-4bdf-9c2c-1ea767189179 response_code = 200 uri = /vpc-routes
19412024-09-01T21:06:43.675ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 157 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:50901 req_id = 553f44fd-0585-4744-9d59-7b14c00cc456 response_code = 204 uri = /vpc-routes
19422024-09-01T21:06:43.828ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:54154
19432024-09-01T21:06:43.828ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
19442024-09-01T21:06:43.868ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
19452024-09-01T21:06:43.929ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 100420 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:54154 req_id = c7883a98-24ef-40d4-969b-3451b29f155d response_code = 204 uri = /switch-ports
19462024-09-01T21:06:43.929ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:43701
19472024-09-01T21:06:43.929ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:37276
19482024-09-01T21:06:43.929ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:58651
19492024-09-01T21:06:43.929ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:50162
19502024-09-01T21:06:43.929ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
19512024-09-01T21:06:43.929ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 356 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:54154 req_id = 2b7f711b-896f-44ea-b32d-4fe928f0ec86 response_code = 200 uri = /network-bootstore-config
19522024-09-01T21:06:43.933ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 4001 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:43701 req_id = a80e6f71-2f91-4693-ad40-81ceac0ddb71 response_code = 200 uri = /v2p
19532024-09-01T21:06:43.934ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3951 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:50162 req_id = f67d182f-5422-456e-a945-78af84f78935 response_code = 200 uri = /vmms/93dbd91d-c521-444f-a889-3e26eca48571/state
19542024-09-01T21:06:43.934ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3903 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:58651 req_id = 9eaccf39-82a2-4217-b500-445f39109fa2 response_code = 200 uri = /v2p
19552024-09-01T21:06:43.935ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte6", ip: 172.30.0.5, mac: MacAddr6([168, 64, 37, 253, 94, 45]), slot: 0, vni: Vni { inner: 5878965 }, subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.0.1 } }) }
19562024-09-01T21:06:43.935ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
19572024-09-01T21:06:43.966ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
19582024-09-01T21:06:43.975ZINFOSledAgent (InstanceManager): Configuring new Omicron zone: oxz_propolis-server_93dbd91d-c521-444f-a889-3e26eca48571 file = illumos-utils/src/zone.rs:316 instance_id = e833a082-213e-41ac-a531-e595eca5375c propolis_id = 93dbd91d-c521-444f-a889-3e26eca48571
19592024-09-01T21:06:43.999ZINFOSledAgent (InstanceManager): Installing Omicron zone: oxz_propolis-server_93dbd91d-c521-444f-a889-3e26eca48571 file = illumos-utils/src/zone.rs:354 instance_id = e833a082-213e-41ac-a531-e595eca5375c propolis_id = 93dbd91d-c521-444f-a889-3e26eca48571
19602024-09-01T21:06:44.030ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 100687 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:37276 req_id = d4115340-2954-4768-9736-5eb2863cc0a0 response_code = 204 uri = /switch-ports
19612024-09-01T21:06:44.031ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 702 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:37276 req_id = e6a4fc1d-889c-4bc9-8f68-84f7ffe6550e response_code = 200 uri = /network-bootstore-config
19622024-09-01T21:06:45.337ZINFOSledAgent (InstanceManager): Profile for oxz_propolis-server_93dbd91d-c521-444f-a889-3e26eca48571: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="system/illumos/propolis-server"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlInstance0'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::1:0'/> <propval type="astring" name="listen_port" value='12400'/> <propval type="astring" name="metric_addr" value='dns'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34 instance_id = e833a082-213e-41ac-a531-e595eca5375c propolis_id = 93dbd91d-c521-444f-a889-3e26eca48571
19632024-09-01T21:06:45.341ZINFOSledAgent (InstanceManager): Booting oxz_propolis-server_93dbd91d-c521-444f-a889-3e26eca48571 zone file = illumos-utils/src/running_zone.rs:488 instance_id = e833a082-213e-41ac-a531-e595eca5375c propolis_id = 93dbd91d-c521-444f-a889-3e26eca48571 zone = oxz_propolis-server_93dbd91d-c521-444f-a889-3e26eca48571
19642024-09-01T21:06:45.522ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_93dbd91d-c521-444f-a889-3e26eca48571") failed: Property not found. retry in 56.970827ms file = illumos-utils/src/svc.rs:36 instance_id = e833a082-213e-41ac-a531-e595eca5375c propolis_id = 93dbd91d-c521-444f-a889-3e26eca48571 zone = oxz_propolis-server_93dbd91d-c521-444f-a889-3e26eca48571
19652024-09-01T21:06:45.594ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_93dbd91d-c521-444f-a889-3e26eca48571") failed: Property not found. retry in 130.913553ms file = illumos-utils/src/svc.rs:36 instance_id = e833a082-213e-41ac-a531-e595eca5375c propolis_id = 93dbd91d-c521-444f-a889-3e26eca48571 zone = oxz_propolis-server_93dbd91d-c521-444f-a889-3e26eca48571
19662024-09-01T21:06:45.744ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_93dbd91d-c521-444f-a889-3e26eca48571") failed: Property not found. retry in 258.696627ms file = illumos-utils/src/svc.rs:36 instance_id = e833a082-213e-41ac-a531-e595eca5375c propolis_id = 93dbd91d-c521-444f-a889-3e26eca48571 zone = oxz_propolis-server_93dbd91d-c521-444f-a889-3e26eca48571
19672024-09-01T21:06:46.018ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_93dbd91d-c521-444f-a889-3e26eca48571") failed: Property not found. retry in 513.697757ms file = illumos-utils/src/svc.rs:36 instance_id = e833a082-213e-41ac-a531-e595eca5375c propolis_id = 93dbd91d-c521-444f-a889-3e26eca48571 zone = oxz_propolis-server_93dbd91d-c521-444f-a889-3e26eca48571
19682024-09-01T21:06:46.542ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_93dbd91d-c521-444f-a889-3e26eca48571") failed: Property not found. retry in 444.755906ms file = illumos-utils/src/svc.rs:36 instance_id = e833a082-213e-41ac-a531-e595eca5375c propolis_id = 93dbd91d-c521-444f-a889-3e26eca48571 zone = oxz_propolis-server_93dbd91d-c521-444f-a889-3e26eca48571
19692024-09-01T21:06:46.999ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_93dbd91d-c521-444f-a889-3e26eca48571") failed: Property not found. retry in 1.107141434s file = illumos-utils/src/svc.rs:36 instance_id = e833a082-213e-41ac-a531-e595eca5375c propolis_id = 93dbd91d-c521-444f-a889-3e26eca48571 zone = oxz_propolis-server_93dbd91d-c521-444f-a889-3e26eca48571
19702024-09-01T21:06:47.943ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 6603 local_addr = [fd00:1122:3344:101::1]:46126 method = GET remote_addr = [fd00:1122:3344:101::d]:42698 req_id = 5e1f52ab-28d4-4609-bdc4-878682add12f response_code = 200 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748 uri = /d40a3584-3bdb-438a-9e24-58fb6b212748
19712024-09-01T21:06:48.118ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_93dbd91d-c521-444f-a889-3e26eca48571") failed: Property not found. retry in 1.445975089s file = illumos-utils/src/svc.rs:36 instance_id = e833a082-213e-41ac-a531-e595eca5375c propolis_id = 93dbd91d-c521-444f-a889-3e26eca48571 zone = oxz_propolis-server_93dbd91d-c521-444f-a889-3e26eca48571
19722024-09-01T21:06:49.574ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_93dbd91d-c521-444f-a889-3e26eca48571") failed: Property not found. retry in 1.026386896s file = illumos-utils/src/svc.rs:36 instance_id = e833a082-213e-41ac-a531-e595eca5375c propolis_id = 93dbd91d-c521-444f-a889-3e26eca48571 zone = oxz_propolis-server_93dbd91d-c521-444f-a889-3e26eca48571
19732024-09-01T21:06:50.630ZINFOSledAgent (InstanceManager): Started propolis in zone: oxz_propolis-server_93dbd91d-c521-444f-a889-3e26eca48571 file = sled-agent/src/instance.rs:1449 instance_id = e833a082-213e-41ac-a531-e595eca5375c propolis_id = 93dbd91d-c521-444f-a889-3e26eca48571
19742024-09-01T21:06:50.638ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_93dbd91d-c521-444f-a889-3e26eca48571") failed: Property not found. retry in 53.480085ms file = illumos-utils/src/svc.rs:36 instance_id = e833a082-213e-41ac-a531-e595eca5375c propolis_id = 93dbd91d-c521-444f-a889-3e26eca48571
19752024-09-01T21:06:50.702ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_93dbd91d-c521-444f-a889-3e26eca48571") failed: Property not found. retry in 74.160318ms file = illumos-utils/src/svc.rs:36 instance_id = e833a082-213e-41ac-a531-e595eca5375c propolis_id = 93dbd91d-c521-444f-a889-3e26eca48571
19762024-09-01T21:06:50.787ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_93dbd91d-c521-444f-a889-3e26eca48571") failed: Property not found. retry in 118.37903ms file = illumos-utils/src/svc.rs:36 instance_id = e833a082-213e-41ac-a531-e595eca5375c propolis_id = 93dbd91d-c521-444f-a889-3e26eca48571
19772024-09-01T21:06:50.917ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_93dbd91d-c521-444f-a889-3e26eca48571") failed: Property not found. retry in 507.95611ms file = illumos-utils/src/svc.rs:36 instance_id = e833a082-213e-41ac-a531-e595eca5375c propolis_id = 93dbd91d-c521-444f-a889-3e26eca48571
19782024-09-01T21:06:51.435ZINFOSledAgent (InstanceManager): Propolis SMF service is online file = sled-agent/src/instance.rs:1458 instance_id = e833a082-213e-41ac-a531-e595eca5375c propolis_id = 93dbd91d-c521-444f-a889-3e26eca48571
19792024-09-01T21:06:51.473ZINFOSledAgent (InstanceManager): Propolis HTTP server online file = sled-agent/src/instance.rs:1482 instance_id = e833a082-213e-41ac-a531-e595eca5375c propolis_id = 93dbd91d-c521-444f-a889-3e26eca48571
19802024-09-01T21:06:51.613ZINFOSledAgent (InstanceManager): result of instance_ensure call is Ok(InstanceEnsureResponse { migrate: None }) file = sled-agent/src/instance.rs:727 instance_id = e833a082-213e-41ac-a531-e595eca5375c propolis_id = 93dbd91d-c521-444f-a889-3e26eca48571
19812024-09-01T21:06:51.614ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Creating), migration_in: None, migration_out: None, time: 2024-09-01T21:06:51.614412428Z } file = sled-agent/src/instance.rs:594 instance_id = e833a082-213e-41ac-a531-e595eca5375c propolis_id = 93dbd91d-c521-444f-a889-3e26eca48571
19822024-09-01T21:06:51.614ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:616 instance_id = e833a082-213e-41ac-a531-e595eca5375c new_vmm_state = VmmRuntimeState { state: Starting, gen: Generation(3), time_updated: 2024-09-01T21:06:51.614412428Z } propolis_id = 93dbd91d-c521-444f-a889-3e26eca48571
19832024-09-01T21:06:51.614ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:514 instance_id = e833a082-213e-41ac-a531-e595eca5375c propolis_id = 93dbd91d-c521-444f-a889-3e26eca48571 state = SledVmmState { vmm_state: VmmRuntimeState { state: Starting, gen: Generation(3), time_updated: 2024-09-01T21:06:51.614412428Z }, migration_in: None, migration_out: None }
19842024-09-01T21:06:51.674ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Running), migration_in: None, migration_out: None, time: 2024-09-01T21:06:51.674374556Z } file = sled-agent/src/instance.rs:594 instance_id = e833a082-213e-41ac-a531-e595eca5375c propolis_id = 93dbd91d-c521-444f-a889-3e26eca48571
19852024-09-01T21:06:51.674ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:616 instance_id = e833a082-213e-41ac-a531-e595eca5375c new_vmm_state = VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2024-09-01T21:06:51.674374556Z } propolis_id = 93dbd91d-c521-444f-a889-3e26eca48571
19862024-09-01T21:06:51.674ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:514 instance_id = e833a082-213e-41ac-a531-e595eca5375c propolis_id = 93dbd91d-c521-444f-a889-3e26eca48571 state = SledVmmState { vmm_state: VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2024-09-01T21:06:51.674374556Z }, migration_in: None, migration_out: None }
19872024-09-01T21:07:12.021ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:53219
19882024-09-01T21:07:12.021ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 46 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:53219 req_id = 5b120b4b-7a07-41d7-9bed-053a96468b92 response_code = 200 uri = /baseboard
19892024-09-01T21:07:12.582ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:56473
19902024-09-01T21:07:12.582ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 175 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:56473 req_id = b714b223-929c-47c2-8089-176bb30dfaa3 response_code = 200 uri = /v2p
19912024-09-01T21:07:12.595ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:35786
19922024-09-01T21:07:12.596ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 730 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:35786 req_id = a5730fd6-23d3-4af9-b98c-a984b7b7cd93 response_code = 200 uri = /vpc-routes
19932024-09-01T21:07:12.675ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:63437
19942024-09-01T21:07:12.675ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 118 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:63437 req_id = 1a6c0a4f-08db-4f96-8d98-6f7df0366ffe response_code = 200 uri = /vmms/93dbd91d-c521-444f-a889-3e26eca48571/state
19952024-09-01T21:07:12.726ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:55321
19962024-09-01T21:07:12.727ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
19972024-09-01T21:07:12.759ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 854 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:35786 req_id = d3bad8d6-d2c1-430f-8259-752c1ca85fe5 response_code = 204 uri = /vpc-routes
19982024-09-01T21:07:12.767ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
19992024-09-01T21:07:12.820ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 93206 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:55321 req_id = 17af136d-a75d-4ec6-ae3a-89eeae7f0153 response_code = 204 uri = /switch-ports
20002024-09-01T21:07:12.820ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 197 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:55321 req_id = 7712f884-79d5-43c9-ad36-11c883466d8a response_code = 200 uri = /network-bootstore-config
20012024-09-01T21:07:13.548ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:35133
20022024-09-01T21:07:13.548ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 54 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:35133 req_id = 4ca039d7-c77f-4941-8bdd-61c91dfcc8bc response_code = 200 uri = /vpc-routes
20032024-09-01T21:07:13.549ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:35946
20042024-09-01T21:07:13.549ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 37 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:35946 req_id = ce3613d7-d76b-4908-9778-ffff181b5320 response_code = 200 uri = /vpc-routes
20052024-09-01T21:07:13.549ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:65135
20062024-09-01T21:07:13.549ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 69 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:65135 req_id = 17ff76f0-fa80-409f-b2fb-c0637afa1984 response_code = 200 uri = /vmms/93dbd91d-c521-444f-a889-3e26eca48571/state
20072024-09-01T21:07:13.569ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 161 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:35946 req_id = ecbbdb32-4278-4f15-8f2a-670f3e34fe6a response_code = 204 uri = /vpc-routes
20082024-09-01T21:07:13.647ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:48339
20092024-09-01T21:07:13.647ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 109 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:48339 req_id = 76f33b79-172d-4d2d-9287-2efa692af752 response_code = 200 uri = /vmms/93dbd91d-c521-444f-a889-3e26eca48571/state
20102024-09-01T21:07:13.651ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:47287
20112024-09-01T21:07:13.651ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 171 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:47287 req_id = 65aa0c67-d310-461e-a53d-daf65fce2687 response_code = 200 uri = /v2p
20122024-09-01T21:07:13.664ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:50677
20132024-09-01T21:07:13.665ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
20142024-09-01T21:07:13.703ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
20152024-09-01T21:07:13.755ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 90845 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:50677 req_id = cbaff8b4-f141-4216-9453-bf20feab1f71 response_code = 204 uri = /switch-ports
20162024-09-01T21:07:13.756ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:35539
20172024-09-01T21:07:13.757ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1003 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:50677 req_id = 8c748b0d-4cbd-421e-9d3e-96c6b96830e2 response_code = 200 uri = /network-bootstore-config
20182024-09-01T21:07:13.757ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1067 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:35133 req_id = d9ac2a06-d6a6-499e-9e91-6fc8a145cd98 response_code = 204 uri = /vpc-routes
20192024-09-01T21:07:13.757ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1103 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:35539 req_id = 9ecca3bf-cdce-4484-ad4a-f4e7b56dd89d response_code = 200 uri = /v2p
20202024-09-01T21:07:13.762ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:61269
20212024-09-01T21:07:13.762ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4038
20222024-09-01T21:07:13.796ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4062
20232024-09-01T21:07:13.848ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 86276 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:61269 req_id = 8dcaf5bb-7b83-4b6c-8db2-c0c60334ae52 response_code = 204 uri = /switch-ports
20242024-09-01T21:07:13.849ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 90 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:61269 req_id = 51db2211-dcbd-4c82-bc92-d44abd1c229c response_code = 200 uri = /network-bootstore-config
20252024-09-01T21:07:17.648ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:63434
20262024-09-01T21:07:17.648ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 586 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:63434 req_id = 1c20d20b-c0a0-458a-aaba-f07bb0c64e37 response_code = 200 uri = /vmms/93dbd91d-c521-444f-a889-3e26eca48571/state
20272024-09-01T21:07:17.648ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Stopping), migration_in: None, migration_out: None, time: 2024-09-01T21:07:17.648963538Z } file = sled-agent/src/instance.rs:594 instance_id = e833a082-213e-41ac-a531-e595eca5375c propolis_id = 93dbd91d-c521-444f-a889-3e26eca48571
20282024-09-01T21:07:17.649ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:616 instance_id = e833a082-213e-41ac-a531-e595eca5375c new_vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2024-09-01T21:07:17.648963538Z } propolis_id = 93dbd91d-c521-444f-a889-3e26eca48571
20292024-09-01T21:07:17.649ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:514 instance_id = e833a082-213e-41ac-a531-e595eca5375c propolis_id = 93dbd91d-c521-444f-a889-3e26eca48571 state = SledVmmState { vmm_state: VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2024-09-01T21:07:17.648963538Z }, migration_in: None, migration_out: None }
20302024-09-01T21:07:17.697ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Destroyed), migration_in: None, migration_out: None, time: 2024-09-01T21:07:17.697827567Z } file = sled-agent/src/instance.rs:594 instance_id = e833a082-213e-41ac-a531-e595eca5375c propolis_id = 93dbd91d-c521-444f-a889-3e26eca48571
20312024-09-01T21:07:17.697ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:616 instance_id = e833a082-213e-41ac-a531-e595eca5375c new_vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(7), time_updated: 2024-09-01T21:07:17.697827567Z } propolis_id = 93dbd91d-c521-444f-a889-3e26eca48571
20322024-09-01T21:07:17.697ZINFOSledAgent (InstanceManager): terminating VMM that has exited file = sled-agent/src/instance.rs:630 instance_id = e833a082-213e-41ac-a531-e595eca5375c propolis_id = 93dbd91d-c521-444f-a889-3e26eca48571
20332024-09-01T21:07:17.697ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
20342024-09-01T21:07:17.697ZWARNSledAgent (kstat-sampler-worker): target removed with queued samples file = oximeter/instruments/src/kstat/sampler.rs:557 id = 17492222599678890273 n_samples = 12 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
20352024-09-01T21:07:17.698ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
20362024-09-01T21:07:17.698ZINFOSledAgent (ZoneBundler): creating zone bundle context = ZoneBundleContext { storage_dirs: ["/pool/int/c13a4209-582e-438f-b076-515144773dc5/debug/bundle/zone", "/pool/int/fdbd95ef-3e32-4c52-bbee-b7e7f0602a6e/debug/bundle/zone"], cause: TerminatedInstance, extra_log_dirs: ["/pool/ext/067f6720-0053-47f5-be0b-12fc815674a2/crypt/debug", "/pool/ext/c65e07fa-2bd7-44d6-b309-46d959ab18fb/crypt/debug", "/pool/ext/e8cbb923-594d-495a-af32-03297bdb9d06/crypt/debug", "/pool/ext/b7b09d01-4b5a-46f4-a346-2e7b6a0e3462/crypt/debug", "/pool/ext/14681c40-9950-4e30-8f61-ec7a2ca60e1c/crypt/debug", "/pool/ext/8d3c930e-5dba-4325-82af-75dd024bc8cc/crypt/debug", "/pool/ext/6421be57-8f17-44b8-8266-2192ac87bae0/crypt/debug", "/pool/ext/64079151-57af-4617-9647-1f77fe70ee41/crypt/debug", "/pool/ext/7a19116b-f2a7-4643-8d6b-df9db59b345a/crypt/debug"] } file = sled-agent/src/zone_bundle.rs:339 zone_name = oxz_propolis-server_93dbd91d-c521-444f-a889-3e26eca48571
20372024-09-01T21:07:17.698ZWARNSledAgent (kstat-sampler-worker): target removed with queued samples file = oximeter/instruments/src/kstat/sampler.rs:557 id = 2731257069035302924 n_samples = 12 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748
20382024-09-01T21:07:17.947ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 11111 local_addr = [fd00:1122:3344:101::1]:46126 method = GET remote_addr = [fd00:1122:3344:101::d]:42698 req_id = d44175e0-8632-4fc2-8bed-2976c3f095d8 response_code = 200 sled_id = d40a3584-3bdb-438a-9e24-58fb6b212748 uri = /d40a3584-3bdb-438a-9e24-58fb6b212748
20392024-09-01T21:07:18.329ZINFOSledAgent (ZoneBundler): finished zone bundle file = sled-agent/src/zone_bundle.rs:1137 metadata = ZoneBundleMetadata { id: ZoneBundleId { zone_name: "oxz_propolis-server_93dbd91d-c521-444f-a889-3e26eca48571", bundle_id: 6b16418f-57de-45e4-80e8-71786493544b }, time_created: 2024-09-01T21:07:17.698751259Z, version: 0, cause: TerminatedInstance }
20402024-09-01T21:07:18.329ZWARNSledAgent (InstanceManager): Halting and removing zone: oxz_propolis-server_93dbd91d-c521-444f-a889-3e26eca48571 file = sled-agent/src/instance.rs:826 instance_id = e833a082-213e-41ac-a531-e595eca5375c propolis_id = 93dbd91d-c521-444f-a889-3e26eca48571
20412024-09-01T21:07:20.046ZINFOSledAgent (InstanceManager): halt_and_remove_logged: Previous zone state: Running file = illumos-utils/src/zone.rs:270 instance_id = e833a082-213e-41ac-a531-e595eca5375c propolis_id = 93dbd91d-c521-444f-a889-3e26eca48571
20422024-09-01T21:07:20.063ZINFOSledAgent (PortManager): Removed route set for subnet file = illumos-utils/src/opte/port_manager.rs:904 id = RouterId { vni: Vni(5878965), kind: System }
20432024-09-01T21:07:20.063ZINFOSledAgent (PortManager): Removed route set for subnet file = illumos-utils/src/opte/port_manager.rs:904 id = RouterId { vni: Vni(5878965), kind: Custom(V4(Ipv4Net { addr: 172.30.0.0, width: 22 })) }
20442024-09-01T21:07:20.063ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:514 instance_id = e833a082-213e-41ac-a531-e595eca5375c propolis_id = 93dbd91d-c521-444f-a889-3e26eca48571 state = SledVmmState { vmm_state: VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2024-09-01T21:07:20.063890831Z }, migration_in: None, migration_out: None }
20452024-09-01T21:07:20.092ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:514 instance_id = e833a082-213e-41ac-a531-e595eca5375c propolis_id = 93dbd91d-c521-444f-a889-3e26eca48571 state = SledVmmState { vmm_state: VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2024-09-01T21:07:20.063890831Z }, migration_in: None, migration_out: None }
20462024-09-01T21:07:20.096ZINFOSledAgent (InstanceManager): State monitoring task complete file = sled-agent/src/instance.rs:763 instance_id = e833a082-213e-41ac-a531-e595eca5375c propolis_id = 93dbd91d-c521-444f-a889-3e26eca48571
20472024-09-01T21:07:20.512ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:35887
20482024-09-01T21:07:20.512ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 47 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:35887 req_id = d549da43-a759-4eed-b3a4-46bdb9cb198c response_code = 200 uri = /vpc-routes
20492024-09-01T21:07:20.521ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 668 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:35887 req_id = e939d1c9-08a9-478b-8cf6-7e6dcfeeb2af response_code = 204 uri = /vpc-routes
20502024-09-01T21:07:20.652ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:37877
20512024-09-01T21:07:20.652ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 121 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:37877 req_id = 2e553a56-112d-455e-af52-6e2cb9d11495 response_code = 200 uri = /v2p
20522024-09-01T21:07:21.124ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:60216
20532024-09-01T21:07:21.125ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 180 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:60216 req_id = 79812841-09f1-400d-8ca6-234f85c970ed response_code = 200 uri = /v2p
20542024-09-01T21:07:22.263ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:40659
20552024-09-01T21:07:22.263ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 114 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:40659 req_id = 860f6d02-1c0e-4513-98a0-4cbcf42d5866 response_code = 200 uri = /vpc-routes
20562024-09-01T21:07:22.271ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 136 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:40659 req_id = 14521874-ee65-49fc-9c91-17de2c6ba911 response_code = 204 uri = /vpc-routes