Max level shown:
1 [ Sep 6 23:34:58 Enabled. ]
2 [ Sep 6 23:34:58 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/sled-agent/sled-agent run /opt/oxide/sled-agent/pkg/config.toml &"). ]
3 [ Sep 6 23:34:58 Method "start" exited with status 0. ]
4 [ Sep 6 23:34:58 Rereading configuration. ]
5 [ Sep 6 23:34:58 No 'refresh' method defined. Treating as :true. ]
6 note: configured to log to "/dev/stdout"
72024-09-06T23:34:58.250ZINFOSledAgent: Ensuring contract reaper thread file = illumos-utils/src/running_zone.rs:113
82024-09-06T23:34:58.250ZINFOSledAgent (BootstrapAgentStartup): finding links ["net0", "net1"] file = sled-agent/src/bootstrap/pre_server.rs:255
92024-09-06T23:34:58.302ZINFOSledAgent (BootstrapAgentStartup): Starting mg-ddm service addr-objs = [AddrObject { interface: "net0", name: "ll" }, AddrObject { interface: "net1", name: "ll" }] file = sled-agent/src/bootstrap/pre_server.rs:262
102024-09-06T23:34:58.302ZINFOSledAgent (BootstrapAgentStartup): Importing mg-ddm service file = sled-agent/src/bootstrap/maghemite.rs:42 path = /opt/oxide/mg-ddm/pkg/ddm/manifest.xml
112024-09-06T23:34:58.417ZINFOSledAgent (BootstrapAgentStartup): Setting mg-ddm interfaces file = sled-agent/src/bootstrap/maghemite.rs:50 interfaces = ("net0/ll" "net1/ll")
122024-09-06T23:34:58.426ZINFOSledAgent (BootstrapAgentStartup): Enabling mg-ddm service file = sled-agent/src/bootstrap/maghemite.rs:56
132024-09-06T23:34:58.434ZINFOSledAgent (BootstrapAgentStartup): Ensuring zfs key directory exists file = sled-agent/src/bootstrap/pre_server.rs:272 path = /var/run/oxide/
142024-09-06T23:34:59.037ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fdb0:18c0:4d0d:a02a::, width: 64 }
152024-09-06T23:34:59.177ZINFOSledAgent: Starting KeyManager file = sled-agent/src/long_running_tasks.rs:126
162024-09-06T23:34:59.177ZINFOSledAgent: Starting StorageManager file = sled-agent/src/long_running_tasks.rs:138
172024-09-06T23:34:59.177ZINFOSledAgent: Starting StorageMonitor file = sled-agent/src/long_running_tasks.rs:151
182024-09-06T23:34:59.177ZINFOSledAgent (DumpSetup-worker): DumpSetup poll loop started. file = sled-agent/src/dump_setup.rs:520
192024-09-06T23:34:59.177ZINFOSledAgent: Starting HardwareManager file = sled-agent/src/long_running_tasks.rs:173 nongimlet_observed_disks = [] sled_mode = Scrimlet { asic: SoftNpuZone }
202024-09-06T23:34:59.177ZINFOSledAgent (HardwareManager): Creating HardwareManager file = sled-hardware/src/illumos/mod.rs:716
212024-09-06T23:34:59.180ZINFOSledAgent (HardwareManager): Generated i86pc baseboard Pc { identifier: "sock", model: "i86pc" } file = sled-hardware/src/illumos/mod.rs:623
222024-09-06T23:34:59.180ZWARNSledAgent (HardwareManager): Device is not a Gimlet (i86pc), proceeding with null hardware view file = sled-hardware/src/illumos/mod.rs:765
232024-09-06T23:34:59.180ZINFOSledAgent: Starting HardwareMonitor file = sled-agent/src/long_running_tasks.rs:187
242024-09-06T23:34:59.180ZINFOSledAgent (HardwareManager): Monitoring for hardware updates file = sled-hardware/src/illumos/mod.rs:824
252024-09-06T23:34:59.180ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/m2_0.vdev
262024-09-06T23:34:59.180ZINFOSledAgent (HardwareMonitor): Checking current full hardware snapshot disks = {} file = sled-agent/src/hardware_monitor.rs:271 underlay_network_info = None
272024-09-06T23:34:59.180ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:451
282024-09-06T23:34:59.180ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
292024-09-06T23:34:59.180ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } path = /scratch/m2_0.vdev
302024-09-06T23:34:59.192ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/m2_0.vdev file = sled-hardware/src/disk.rs:360
312024-09-06T23:34:59.192ZINFOSledAgent (StorageResources): Formatting zpool with generated ID file = sled-hardware/src/disk.rs:371 id = ca24db04-f37b-40b1-83c2-23476bec5a4a (zpool)
322024-09-06T23:34:59.288ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:185 zpool = ZpoolName { id: ca24db04-f37b-40b1-83c2-23476bec5a4a (zpool), kind: Internal }
332024-09-06T23:34:59.702ZINFOSledAgent (StorageManager): Received DetectedRawDisksChanged { raw_disks: {}, tx: ... } file = sled-storage/src/manager.rs:451
342024-09-06T23:34:59.702ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/m2_1.vdev
352024-09-06T23:34:59.702ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
362024-09-06T23:34:59.702ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
372024-09-06T23:34:59.702ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:451
382024-09-06T23:34:59.702ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
392024-09-06T23:34:59.702ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" } path = /scratch/m2_1.vdev
402024-09-06T23:34:59.711ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/m2_1.vdev file = sled-hardware/src/disk.rs:360
412024-09-06T23:34:59.711ZINFOSledAgent (StorageResources): Formatting zpool with generated ID file = sled-hardware/src/disk.rs:371 id = 358fc814-3f2a-4b1a-aebb-894434d72d01 (zpool)
422024-09-06T23:34:59.800ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:185 zpool = ZpoolName { id: 358fc814-3f2a-4b1a-aebb-894434d72d01 (zpool), kind: Internal }
432024-09-06T23:35:00.207ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_0.vdev
442024-09-06T23:35:00.207ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: ca24db04-f37b-40b1-83c2-23476bec5a4a (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 358fc814-3f2a-4b1a-aebb-894434d72d01 (zpool), kind: Internal } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
452024-09-06T23:35:00.207ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
462024-09-06T23:35:00.207ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
472024-09-06T23:35:00.207ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:451
482024-09-06T23:35:00.207ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
492024-09-06T23:35:00.207ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_1.vdev
502024-09-06T23:35:00.207ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: ca24db04-f37b-40b1-83c2-23476bec5a4a (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 358fc814-3f2a-4b1a-aebb-894434d72d01 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
512024-09-06T23:35:00.207ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:451
522024-09-06T23:35:00.207ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
532024-09-06T23:35:00.207ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
542024-09-06T23:35:00.207ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
552024-09-06T23:35:00.207ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_2.vdev
562024-09-06T23:35:00.207ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: ca24db04-f37b-40b1-83c2-23476bec5a4a (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 358fc814-3f2a-4b1a-aebb-894434d72d01 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
572024-09-06T23:35:00.207ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:451
582024-09-06T23:35:00.207ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
592024-09-06T23:35:00.207ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
602024-09-06T23:35:00.207ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
612024-09-06T23:35:00.207ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_3.vdev
622024-09-06T23:35:00.207ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: ca24db04-f37b-40b1-83c2-23476bec5a4a (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 358fc814-3f2a-4b1a-aebb-894434d72d01 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
632024-09-06T23:35:00.207ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
642024-09-06T23:35:00.207ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:451
652024-09-06T23:35:00.207ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
662024-09-06T23:35:00.207ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
672024-09-06T23:35:00.207ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_4.vdev
682024-09-06T23:35:00.207ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: ca24db04-f37b-40b1-83c2-23476bec5a4a (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 358fc814-3f2a-4b1a-aebb-894434d72d01 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
692024-09-06T23:35:00.207ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
702024-09-06T23:35:00.207ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
712024-09-06T23:35:00.207ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:451
722024-09-06T23:35:00.207ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
732024-09-06T23:35:00.207ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_5.vdev
742024-09-06T23:35:00.207ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: ca24db04-f37b-40b1-83c2-23476bec5a4a (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 358fc814-3f2a-4b1a-aebb-894434d72d01 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
752024-09-06T23:35:00.207ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
762024-09-06T23:35:00.207ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
772024-09-06T23:35:00.207ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:451
782024-09-06T23:35:00.207ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
792024-09-06T23:35:00.207ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_6.vdev
802024-09-06T23:35:00.207ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: ca24db04-f37b-40b1-83c2-23476bec5a4a (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 358fc814-3f2a-4b1a-aebb-894434d72d01 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
812024-09-06T23:35:00.207ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
822024-09-06T23:35:00.207ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
832024-09-06T23:35:00.207ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:451
842024-09-06T23:35:00.207ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
852024-09-06T23:35:00.207ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_7.vdev
862024-09-06T23:35:00.207ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: ca24db04-f37b-40b1-83c2-23476bec5a4a (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 358fc814-3f2a-4b1a-aebb-894434d72d01 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
872024-09-06T23:35:00.207ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
882024-09-06T23:35:00.207ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
892024-09-06T23:35:00.207ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:451
902024-09-06T23:35:00.207ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
912024-09-06T23:35:00.207ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_8.vdev
922024-09-06T23:35:00.207ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:451
932024-09-06T23:35:00.207ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: ca24db04-f37b-40b1-83c2-23476bec5a4a (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 358fc814-3f2a-4b1a-aebb-894434d72d01 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
942024-09-06T23:35:00.207ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
952024-09-06T23:35:00.207ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
962024-09-06T23:35:00.207ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
972024-09-06T23:35:00.207ZINFOSledAgent: Waiting for boot disk file = sled-agent/src/long_running_tasks.rs:97
982024-09-06T23:35:00.207ZINFOSledAgent: Found boot disk DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } file = sled-agent/src/long_running_tasks.rs:99
992024-09-06T23:35:00.207ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: ca24db04-f37b-40b1-83c2-23476bec5a4a (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 358fc814-3f2a-4b1a-aebb-894434d72d01 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
1002024-09-06T23:35:00.207ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
1012024-09-06T23:35:00.207ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
1022024-09-06T23:35:00.207ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
1032024-09-06T23:35:00.207ZINFOSledAgent: Starting Bootstore file = sled-agent/src/long_running_tasks.rs:211
1042024-09-06T23:35:00.207ZINFOSledAgent (bootstore): No ledger in /pool/int/ca24db04-f37b-40b1-83c2-23476bec5a4a/cluster/bootstore-fsm-state.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1052024-09-06T23:35:00.207ZINFOSledAgent (bootstore): No ledger in /pool/int/358fc814-3f2a-4b1a-aebb-894434d72d01/cluster/bootstore-fsm-state.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1062024-09-06T23:35:00.207ZINFOSledAgent (bootstore): No ledger found. Loading Fsm::State as Uninitialized file = bootstore/src/schemes/v0/storage.rs:83 peer_id = pc-sock-i86pc
1072024-09-06T23:35:00.207ZINFOSledAgent (bootstore): No ledger in /pool/int/ca24db04-f37b-40b1-83c2-23476bec5a4a/cluster/bootstore-network-config.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1082024-09-06T23:35:00.207ZINFOSledAgent (bootstore): No ledger in /pool/int/358fc814-3f2a-4b1a-aebb-894434d72d01/cluster/bootstore-network-config.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1092024-09-06T23:35:00.207ZINFOSledAgent (bootstore): No ledger found for network config file = bootstore/src/schemes/v0/storage.rs:146 peer_id = pc-sock-i86pc
1102024-09-06T23:35:00.207ZINFOSledAgent: Starting Bootstore DDMD poller file = sled-agent/src/long_running_tasks.rs:216
1112024-09-06T23:35:00.207ZINFOSledAgent: Starting ZoneBundler related tasks file = sled-agent/src/long_running_tasks.rs:231
1122024-09-06T23:35:00.218ZINFOSledAgent (ServiceManager): Creating ServiceManager file = sled-agent/src/services.rs:730
1132024-09-06T23:35:00.218ZINFOSledAgent (auto-cleanup-task): top of bundle cleanup loop file = sled-agent/src/zone_bundle.rs:186 next_cleanup = Instant { tv_sec: 2127, tv_nsec: 668364464 } time_to_next_cleanup = 599.999919939s
1142024-09-06T23:35:00.218ZINFOSledAgent (ServiceManager): Ensuring scrimlet services (enabling services) file = sled-agent/src/services.rs:3842
1152024-09-06T23:35:00.218ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
1162024-09-06T23:35:00.218ZINFOSledAgent (BootstrapAgentStartup): No ledger in /pool/int/ca24db04-f37b-40b1-83c2-23476bec5a4a/config/sled-agent-request.json file = common/src/ledger.rs:188
1172024-09-06T23:35:00.218ZINFOSledAgent (BootstrapAgentStartup): No ledger in /pool/int/358fc814-3f2a-4b1a-aebb-894434d72d01/config/sled-agent-request.json file = common/src/ledger.rs:188
1182024-09-06T23:35:00.221ZINFOSledAgent (dropshot (BootstrapAgent)): listening file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:205 local_addr = [fdb0:18c0:4d0d:a02a::1]:80
1192024-09-06T23:35:00.221ZINFOSledAgent (SledAgentSprocketsServer): Started listening file = sled-agent/src/bootstrap/sprockets_server.rs:44 local_addr = [fdb0:18c0:4d0d:a02a::1]:12346
1202024-09-06T23:35:00.221ZINFOSledAgent (RSS): Injecting RSS configuration: RackInitializeRequest { trust_quorum_peers: None, bootstrap_discovery: OnlyOurs, ntp_servers: [ "0.pool.ntp.org", ], dns_servers: [ 1.1.1.1, 9.9.9.9, ], internal_services_ip_pool_ranges: [ V4( Ipv4Range { first: 10.151.2.164, last: 10.151.2.173, }, ), ], external_dns_ips: [ 10.151.2.164, 10.151.2.165, ], external_dns_zone_name: "oxide.test", external_certificates: "<redacted>", recovery_silo: RecoverySiloConfig { silo_name: Name( "recovery", ), user_name: UserId( "recovery", ), user_password_hash: NewPasswordHash( PasswordHashString { string: "$argon2id$v=19$m=98304,t=13,p=1$RUlWc0ZxaHo0WFdrN0N6ZQ$S8p52j85GPvMhR/ek3GL0el/oProgTwWpHJZ8lsQQoY", encoding: B64, }, ), }, rack_network_config: RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [ PortConfigV2 { routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), nexthop: 10.151.2.100, vlan_id: None, local_pref: None, }, ], addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.2.174, width: 24, }, ), vlan_id: None, }, ], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: None, bgp_peers: [], autoneg: false, lldp: None, }, ], bgp: [], bfd: [], }, allowed_source_ips: Any, } file = sled-agent/src/rack_setup/service.rs:1074
1212024-09-06T23:35:00.221ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = internal-dns/src/resolver.rs:73
1222024-09-06T23:35:00.221ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
1232024-09-06T23:35:00.221ZINFOSledAgent (RSS): No ledger in /pool/int/ca24db04-f37b-40b1-83c2-23476bec5a4a/config/rss-plan-completed.marker file = common/src/ledger.rs:188
1242024-09-06T23:35:00.221ZINFOSledAgent (RSS): No ledger in /pool/int/358fc814-3f2a-4b1a-aebb-894434d72d01/config/rss-plan-completed.marker file = common/src/ledger.rs:188
1252024-09-06T23:35:00.221ZINFOSledAgent (RSS): RSS configuration has not been fully applied yet file = sled-agent/src/rack_setup/service.rs:1137
1262024-09-06T23:35:00.221ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
1272024-09-06T23:35:00.221ZINFOSledAgent (RSS): No ledger in /pool/int/ca24db04-f37b-40b1-83c2-23476bec5a4a/config/rss-sled-plan.json file = common/src/ledger.rs:188
1282024-09-06T23:35:00.221ZINFOSledAgent (RSS): No ledger in /pool/int/358fc814-3f2a-4b1a-aebb-894434d72d01/config/rss-sled-plan.json file = common/src/ledger.rs:188
1292024-09-06T23:35:00.221ZINFOSledAgent (RSS): Creating new allocation plan file = sled-agent/src/rack_setup/service.rs:1179
1302024-09-06T23:35:00.221ZINFOSledAgent (RSS): Serializing plan file = sled-agent/src/rack_setup/plan/sled.rs:155
1312024-09-06T23:35:00.221ZINFOSledAgent (RSS): Creating plan for the sled at fdb0:18c0:4d0d:a02a::1 file = sled-agent/src/rack_setup/plan/sled.rs:128
1322024-09-06T23:35:00.221ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
1332024-09-06T23:35:00.221ZINFOSledAgent (RSS): Writing ledger to /pool/int/ca24db04-f37b-40b1-83c2-23476bec5a4a/config/.rss-sled-plan.json.tmp file = common/src/ledger.rs:199
1342024-09-06T23:35:00.222ZINFOSledAgent (RSS): Writing ledger to /pool/int/358fc814-3f2a-4b1a-aebb-894434d72d01/config/.rss-sled-plan.json.tmp file = common/src/ledger.rs:199
1352024-09-06T23:35:00.222ZINFOSledAgent (RSS): Sled plan written to storage: Plan { rack_id: f8e86eb4-0496-4cd3-9bda-2c06ff0605a7, sleds: { [fdb0:18c0:4d0d:a02a::1]:12346: StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: 55267d89-b780-4916-86a8-64533b7bfd1c, rack_id: f8e86eb4-0496-4cd3-9bda-2c06ff0605a7, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64, }, }, }, }, }, config: RackInitializeRequest { trust_quorum_peers: None, bootstrap_discovery: OnlyOurs, ntp_servers: [ "0.pool.ntp.org", ], dns_servers: [ 1.1.1.1, 9.9.9.9, ], internal_services_ip_pool_ranges: [ V4( Ipv4Range { first: 10.151.2.164, last: 10.151.2.173, }, ), ], external_dns_ips: [ 10.151.2.164, 10.151.2.165, ], external_dns_zone_name: "oxide.test", external_certificates: "<redacted>", recovery_silo: RecoverySiloConfig { silo_name: Name( "recovery", ), user_name: UserId( "recovery", ), user_password_hash: NewPasswordHash( PasswordHashString { string: "$argon2id$v=19$m=98304,t=13,p=1$RUlWc0ZxaHo0WFdrN0N6ZQ$S8p52j85GPvMhR/ek3GL0el/oProgTwWpHJZ8lsQQoY", encoding: B64, }, ), }, rack_network_config: RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [ PortConfigV2 { routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), nexthop: 10.151.2.100, vlan_id: None, local_pref: None, }, ], addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.2.174, width: 24, }, ), vlan_id: None, }, ], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: None, bgp_peers: [], autoneg: false, lldp: None, }, ], bgp: [], bfd: [], }, allowed_source_ips: Any, }, } file = sled-agent/src/rack_setup/plan/sled.rs:175
1362024-09-06T23:35:00.222ZINFOSledAgent (RSS): Writing Rack Network Configuration to bootstore file = sled-agent/src/rack_setup/service.rs:1213
1372024-09-06T23:35:00.222ZINFOSledAgent (bootstore): Attempting to update network config with generation: 1, current_generation: 0 file = bootstore/src/schemes/v0/peer.rs:564 peer_id = pc-sock-i86pc
1382024-09-06T23:35:00.222ZINFOSledAgent (bootstore): Writing ledger to /pool/int/ca24db04-f37b-40b1-83c2-23476bec5a4a/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
1392024-09-06T23:35:00.222ZINFOSledAgent (bootstore): Writing ledger to /pool/int/358fc814-3f2a-4b1a-aebb-894434d72d01/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
1402024-09-06T23:35:00.222ZINFOSledAgent (bootstore): Broadcasting network config with generation 1 file = bootstore/src/schemes/v0/peer.rs:648 peer_id = pc-sock-i86pc
1412024-09-06T23:35:00.222ZINFOSledAgent (BootstrapAgentRssHandler): Received initialization request from RSS file = sled-agent/src/bootstrap/rss_handle.rs:221 request = StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: 55267d89-b780-4916-86a8-64533b7bfd1c, rack_id: f8e86eb4-0496-4cd3-9bda-2c06ff0605a7, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64 } } } } target_sled = [fdb0:18c0:4d0d:a02a::1]:12346
1422024-09-06T23:35:00.223ZINFOSledAgent (SledAgentSprocketsServer): Accepted connection file = sled-agent/src/bootstrap/sprockets_server.rs:66 remote_addr = [fdb0:18c0:4d0d:a02a::1]:42437
1432024-09-06T23:35:00.223ZINFOSledAgent (SledAgentMain): Loading Sled Agent: StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: 55267d89-b780-4916-86a8-64533b7bfd1c, rack_id: f8e86eb4-0496-4cd3-9bda-2c06ff0605a7, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64 } } } } file = sled-agent/src/bootstrap/server.rs:359
1442024-09-06T23:35:00.223ZINFOSledAgent (SledAgentMain): KeyManager: using hardcoded secret retriever file = sled-agent/src/bootstrap/server.rs:374
1452024-09-06T23:35:00.223ZINFOSledAgent (StorageManager): Received KeyManagerReady file = sled-storage/src/manager.rs:451
1462024-09-06T23:35:00.223ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:101::, width: 64 }
1472024-09-06T23:35:00.223ZINFOSledAgent: setting up sled agent server file = sled-agent/src/server.rs:44
1482024-09-06T23:35:00.223ZINFOSledAgent (StorageManager): Loading storage config file = sled-storage/src/manager.rs:644
1492024-09-06T23:35:00.223ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = internal-dns/src/resolver.rs:73
1502024-09-06T23:35:00.223ZINFOSledAgent (StorageManager): No ledger in /pool/int/ca24db04-f37b-40b1-83c2-23476bec5a4a/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = load_disks_ledger
1512024-09-06T23:35:00.223ZINFOSledAgent (StorageManager): No ledger in /pool/int/358fc814-3f2a-4b1a-aebb-894434d72d01/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = load_disks_ledger
1522024-09-06T23:35:00.223ZINFOSledAgent (StorageManager): No ledger of physical disks exists file = sled-storage/src/manager.rs:633
1532024-09-06T23:35:00.223ZINFOSledAgent (StorageManager): KeyManager ready, but no ledger detected file = sled-storage/src/manager.rs:661
1542024-09-06T23:35:00.223ZINFOSledAgent (StorageManager): automatically managing disks file = sled-storage/src/manager.rs:440
1552024-09-06T23:35:00.223ZINFOSledAgent (StorageResources): Synchronizing disk managment file = sled-storage/src/resources.rs:302
1562024-09-06T23:35:00.223ZINFOSledAgent (StorageManager): Successfully synchronized disks without error file = sled-storage/src/manager.rs:532 result = DisksManagementResult { status: [] }
1572024-09-06T23:35:00.225ZINFOSledAgent (ServiceManager): Enabling switch zone (new) file = sled-agent/src/services.rs:4139
1582024-09-06T23:35:00.225ZINFOSledAgent (ServiceManager): Starting switch zone file = sled-agent/src/services.rs:4521
1592024-09-06T23:35:00.256ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
1602024-09-06T23:35:00.277ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_switch file = illumos-utils/src/zone.rs:316
1612024-09-06T23:35:00.295ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_switch file = illumos-utils/src/zone.rs:354
1622024-09-06T23:35:00.347ZINFOSledAgent: SledAgent::new(..) starting file = sled-agent/src/sled_agent.rs:390 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c
1632024-09-06T23:35:00.347ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
1642024-09-06T23:35:00.347ZINFOSledAgent: Requested swap device of size 64 GiB file = sled-agent/src/sled_agent.rs:402 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c
1652024-09-06T23:35:00.347ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1662024-09-06T23:35:00.350ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 142.991095ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "14a202f6-5585-4870-9cb8-fcbf70e5fdf4", "content-length": "124", "date": "Fri, 06 Sep 2024 23:35:00 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "14a202f6-5585-4870-9cb8-fcbf70e5fdf4" } file = clients/ddm-admin-client/src/lib.rs:177
1672024-09-06T23:35:00.361ZINFOSledAgent: attempting to create encrypted zvol: name="oxi_ca24db04-f37b-40b1-83c2-23476bec5a4a/swap", size_gb=64 file = sled-agent/src/swap_device.rs:153
1682024-09-06T23:35:00.385ZINFOSledAgent: successfully created encrypted zvol: name="oxi_ca24db04-f37b-40b1-83c2-23476bec5a4a/swap", size_gb=64 file = sled-agent/src/swap_device.rs:249
1692024-09-06T23:35:00.385ZINFOSledAgent: adding swap device: swapname="/dev/zvol/dsk/oxi_ca24db04-f37b-40b1-83c2-23476bec5a4a/swap" file = sled-agent/src/swap_device.rs:96
1702024-09-06T23:35:00.391ZINFOSledAgent: Mounting backing filesystems file = sled-agent/src/sled_agent.rs:417 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c
1712024-09-06T23:35:00.391ZINFOSledAgent (BackingFs): Processing fmd file = sled-agent/src/backing_fs.rs:130
1722024-09-06T23:35:00.391ZINFOSledAgent (BackingFs): Ensuring dataset oxi_ca24db04-f37b-40b1-83c2-23476bec5a4a/backing/fmd file = sled-agent/src/backing_fs.rs:140
1732024-09-06T23:35:00.483ZINFOSledAgent (BackingFs): Stopping service svc:/system/fmd:default file = sled-agent/src/backing_fs.rs:175
1742024-09-06T23:35:00.494ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1752024-09-06T23:35:00.495ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 126.368927ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "0944c10d-44b5-4e8f-a119-75f381024352", "content-length": "124", "date": "Fri, 06 Sep 2024 23:35:00 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "0944c10d-44b5-4e8f-a119-75f381024352" } file = clients/ddm-admin-client/src/lib.rs:177
1762024-09-06T23:35:00.622ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1772024-09-06T23:35:00.622ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 183.071106ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "5540e67c-6a3a-4309-860c-d3132445d4e0", "content-length": "124", "date": "Fri, 06 Sep 2024 23:35:00 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "5540e67c-6a3a-4309-860c-d3132445d4e0" } file = clients/ddm-admin-client/src/lib.rs:177
1782024-09-06T23:35:00.703ZINFOSledAgent (BackingFs): Mounting oxi_ca24db04-f37b-40b1-83c2-23476bec5a4a/backing/fmd on /var/fm/fmd file = sled-agent/src/backing_fs.rs:183
1792024-09-06T23:35:00.716ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/rsrc file = sled-agent/src/backing_fs.rs:191
1802024-09-06T23:35:00.716ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/ckpt file = sled-agent/src/backing_fs.rs:191
1812024-09-06T23:35:00.716ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/xprt file = sled-agent/src/backing_fs.rs:191
1822024-09-06T23:35:00.716ZINFOSledAgent (BackingFs): Starting service svc:/system/fmd:default file = sled-agent/src/backing_fs.rs:197
1832024-09-06T23:35:00.806ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1842024-09-06T23:35:00.807ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 186.231205ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "842bc68c-6215-4807-b14c-a0980e5737b6", "content-length": "124", "date": "Fri, 06 Sep 2024 23:35:00 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "842bc68c-6215-4807-b14c-a0980e5737b6" } file = clients/ddm-admin-client/src/lib.rs:177
1852024-09-06T23:35:00.994ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1862024-09-06T23:35:00.994ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 136.59104ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "96e8cfde-cd4f-450f-b76a-97806e9e771e", "content-length": "124", "date": "Fri, 06 Sep 2024 23:35:00 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "96e8cfde-cd4f-450f-b76a-97806e9e771e" } file = clients/ddm-admin-client/src/lib.rs:177
1872024-09-06T23:35:01.131ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1882024-09-06T23:35:01.132ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 344.703973ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "37ebb45f-aadc-48a3-b40c-83a04942fe29", "content-length": "124", "date": "Fri, 06 Sep 2024 23:35:00 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "37ebb45f-aadc-48a3-b40c-83a04942fe29" } file = clients/ddm-admin-client/src/lib.rs:177
1892024-09-06T23:35:01.459ZINFOSledAgent: using '[AddrObject { interface: "net0", name: "ll" }, AddrObject { interface: "net1", name: "ll" }]' as data links for xde driver file = illumos-utils/src/opte/illumos.rs:93 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c
1902024-09-06T23:35:01.467ZINFOSledAgent (dropshot): listening file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:205 local_addr = [fd00:1122:3344:101::1]:39558 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c
1912024-09-06T23:35:01.467ZINFOSledAgent (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = internal-dns/src/resolver.rs:73 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c
1922024-09-06T23:35:01.467ZINFOSledAgent (producer-server): starting oximeter metric producer server address = [fd00:1122:3344:101::1]:39558 file = oximeter/producer/src/lib.rs:283 interval = 30s producer_id = 55267d89-b780-4916-86a8-64533b7bfd1c sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c
1932024-09-06T23:35:01.469ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 299.218092ms error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c
1942024-09-06T23:35:01.471ZINFOSledAgent (VmmReservoirManager): 50% of 102952026112 physical ram = 51476013056 bytes) file = sled-agent/src/vmm_reservoir.rs:267 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c
1952024-09-06T23:35:01.471ZINFOSledAgent (VmmReservoirManager): Setting reservoir size to 49090 MiB bytes file = sled-agent/src/vmm_reservoir.rs:275 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c
1962024-09-06T23:35:01.471ZINFOSledAgent (ServiceManager): sled agent started file = sled-agent/src/services.rs:914 underlay_address = fd00:1122:3344:101::1
1972024-09-06T23:35:01.471ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = internal-dns/src/resolver.rs:73
1982024-09-06T23:35:01.478ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1992024-09-06T23:35:01.770ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 654.349323ms error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c
2002024-09-06T23:35:02.426ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 636.790452ms error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c
2012024-09-06T23:35:03.065ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 1.461074429s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c
2022024-09-06T23:35:03.887ZINFOSledAgent (ServiceManager): Setting up dendrite service file = sled-agent/src/services.rs:2615
2032024-09-06T23:35:03.887ZINFOSledAgent (ServiceManager): Setting up lldpd service file = sled-agent/src/services.rs:2833
2042024-09-06T23:35:03.887ZINFOSledAgent (ServiceManager): Setting up MGS service file = sled-agent/src/services.rs:2485
2052024-09-06T23:35:03.887ZINFOSledAgent (ServiceManager): Setting up wicketd service file = sled-agent/src/services.rs:2532
2062024-09-06T23:35:03.887ZINFOSledAgent (ServiceManager): Setting up mgd service file = sled-agent/src/services.rs:2914
2072024-09-06T23:35:03.887ZINFOSledAgent (ServiceManager): Setting up mg-ddm service file = sled-agent/src/services.rs:2956
2082024-09-06T23:35:03.889ZINFOSledAgent (ServiceManager): Setting up tfport service file = sled-agent/src/services.rs:2758
2092024-09-06T23:35:03.891ZINFOSledAgent (ServiceManager): Setting up Simulated SP service file = sled-agent/src/services.rs:2526
2102024-09-06T23:35:03.891ZINFOSledAgent (ServiceManager): Profile for oxz_switch: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService0'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/mgs"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="address" value='[::1]:12225'/> <propval type="astring" name="id" value='9930d33b-f363-4dbc-982f-ca5aa7311384'/> <propval type="astring" name="rack_id" value='f8e86eb4-0496-4cd3-9bda-2c06ff0605a7'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/wicketd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="address" value='[::1]:12226'/> <propval type="astring" name="artifact-address" value='[fdb0:18c0:4d0d:a02a::2]:12227'/> <propval type="astring" name="baseboard-file" value='/opt/oxide/baseboard.json'/> <propval type="astring" name="mgs-address" value='[::1]:12225'/> <propval type="astring" name="nexus-proxy-address" value='[::]:12229'/> <propval type="astring" name="rack-subnet" value='fd00:1122:3344::'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/switch_zone_setup"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="gz_local_link_addr" value='fe80::8:20ff:fee8:67d9'/> <property type="astring" name="link_local_links"> <astring_list> <value_node value="tfportrear0_0" /> <value_node value="oxBootstrap0" /> </astring_list> </property> <propval type="astring" name="bootstrap_addr" value='fdb0:18c0:4d0d:a02a::2'/> <propval type="astring" name="bootstrap_vnic" value='oxBootstrap0'/> <propval type="astring" name="baseboard_info" value='{"type":"pc","identifier":"sock","model":"i86pc"}'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/dendrite"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="sled_id" value='55267d89-b780-4916-86a8-64533b7bfd1c'/> <propval type="astring" name="rack_id" value='f8e86eb4-0496-4cd3-9bda-2c06ff0605a7'/> <propval type="astring" name="address" value='[::1]:12224'/> <propval type="astring" name="front_ports" value='1'/> <propval type="astring" name="rear_ports" value='1'/> <propval type="astring" name="port_config" value='/opt/oxide/dendrite/misc/softnpu_single_sled_config.toml'/> <propval type="astring" name="mgmt" value='uds'/> <propval type="astring" name="uds_path" value='/opt/softnpu/stuff'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/tfport"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="host" value='[::1]'/> <propval type="astring" name="port" value='12224'/> <propval type="astring" name="flags" value='--sync-only'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/lldpd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="board_rev" value='softnpu_front_1_rear_1'/> <propval type="astring" name="scrimlet_id" value='sock'/> <propval type="astring" name="scrimlet_model" value='i86pc'/> <propval type="astring" name="address" value='[::1]:12230'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/pumpkind"> </service> <service version="1" type="service" name="oxide/mgd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="sled_uuid" value='55267d89-b780-4916-86a8-64533b7bfd1c'/> <propval type="astring" name="rack_uuid" value='f8e86eb4-0496-4cd3-9bda-2c06ff0605a7'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/mg-ddm"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="mode" value='transit'/> <propval type="astring" name="dendrite" value='true'/> <propval type="astring" name="sled_uuid" value='55267d89-b780-4916-86a8-64533b7bfd1c'/> <propval type="astring" name="rack_uuid" value='f8e86eb4-0496-4cd3-9bda-2c06ff0605a7'/> <propval type="astring" name="interfaces" value='tfportrear0_0/ll'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/uplink"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
2112024-09-06T23:35:03.891ZINFOSledAgent (ServiceManager): Booting oxz_switch zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_switch
2122024-09-06T23:35:04.289ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 51.178465ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2132024-09-06T23:35:04.475ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 55.855801ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2142024-09-06T23:35:04.528ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 2.972988832s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c
2152024-09-06T23:35:04.657ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 230.514809ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2162024-09-06T23:35:05.004ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 236.614356ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2172024-09-06T23:35:05.381ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 993.683046ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2182024-09-06T23:35:06.580ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 905.35573ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2192024-09-06T23:35:07.709ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 623.854667ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2202024-09-06T23:35:07.710ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 6.143382713s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c
2212024-09-06T23:35:08.390ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 789.766577ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2222024-09-06T23:35:09.413ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 1.14066945s file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2232024-09-06T23:35:10.926ZINFOSledAgent (dropshot (SledAgent)): listening file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:205 local_addr = [fd00:1122:3344:101::1]:12345
2242024-09-06T23:35:10.926ZINFOSledAgent (SledAgentMain): Sled Agent loaded; recording configuration file = sled-agent/src/bootstrap/server.rs:429
2252024-09-06T23:35:10.926ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
2262024-09-06T23:35:10.926ZINFOSledAgent (SledAgentMain): Writing ledger to /pool/int/ca24db04-f37b-40b1-83c2-23476bec5a4a/config/.sled-agent-request.json.tmp file = common/src/ledger.rs:199
2272024-09-06T23:35:10.926ZINFOSledAgent (SledAgentMain): Writing ledger to /pool/int/358fc814-3f2a-4b1a-aebb-894434d72d01/config/.sled-agent-request.json.tmp file = common/src/ledger.rs:199
2282024-09-06T23:35:10.926ZINFOSledAgent (HardwareMonitor): Sled Agent Started file = sled-agent/src/hardware_monitor.rs:138
2292024-09-06T23:35:10.926ZINFOSledAgent (SledAgentSprocketsServer): Connection closed file = sled-agent/src/bootstrap/sprockets_server.rs:73 remote_addr = [fdb0:18c0:4d0d:a02a::1]:42437
2302024-09-06T23:35:10.926ZINFOSledAgent (HardwareMonitor): Checking current full hardware snapshot disks = {} file = sled-agent/src/hardware_monitor.rs:271 underlay_network_info = Some((fd00:1122:3344:101::2, Some(RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56 }, infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [PortConfigV2 { routes: [RouteConfig { destination: V4(Ipv4Net { addr: 0.0.0.0, width: 0 }), nexthop: 10.151.2.100, vlan_id: None, local_pref: None }], addresses: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: None, bgp_peers: [], autoneg: false, lldp: None }], bgp: [], bfd: [] })))
2312024-09-06T23:35:10.926ZINFOSledAgent (ServiceManager): Ensuring scrimlet services (enabling services) file = sled-agent/src/services.rs:3842
2322024-09-06T23:35:11.148ZINFOSledAgent (BootstrapAgentRssHandler): Peer agent initialized file = sled-agent/src/bootstrap/rss_handle.rs:112 peer = [fdb0:18c0:4d0d:a02a::1]:12346
2332024-09-06T23:35:11.148ZINFOSledAgent (BootstrapAgentRssHandler): Initialized sled agent file = sled-agent/src/bootstrap/rss_handle.rs:236 target_sled = [fdb0:18c0:4d0d:a02a::1]:12346
2342024-09-06T23:35:11.148ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
2352024-09-06T23:35:11.148ZINFOSledAgent (RSS): No ledger in /pool/int/ca24db04-f37b-40b1-83c2-23476bec5a4a/config/rss-service-plan-v4.json file = common/src/ledger.rs:188
2362024-09-06T23:35:11.148ZINFOSledAgent (RSS): No ledger in /pool/int/358fc814-3f2a-4b1a-aebb-894434d72d01/config/rss-service-plan-v4.json file = common/src/ledger.rs:188
2372024-09-06T23:35:11.148ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
2382024-09-06T23:35:11.148ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
2392024-09-06T23:35:11.148ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
2402024-09-06T23:35:11.153ZINFOSledAgent (ServiceManager): Re-enabling running switch zone (new address) file = sled-agent/src/services.rs:4163 new = [fd00:1122:3344:101::2, ::1] old = [::1]
2412024-09-06T23:35:11.153ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::2 exists file = sled-agent/src/services.rs:4179
2422024-09-06T23:35:11.153ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:101::2, prefix: 64 })) file = illumos-utils/src/running_zone.rs:534 zone = oxz_switch
2432024-09-06T23:35:11.425ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:50521
2442024-09-06T23:35:11.425ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
2452024-09-06T23:35:11.425ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 225 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:50521 req_id = 0cf6fe1c-c616-4941-ba22-1ece69871832 response_code = 200 uri = /inventory
2462024-09-06T23:35:11.456ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:47727
2472024-09-06T23:35:11.456ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 33 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:47727 req_id = 61b102b0-2d52-4a8a-9e78-380782440beb response_code = 200 uri = /sled-role
2482024-09-06T23:35:11.458ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
2492024-09-06T23:35:11.458ZINFOSledAgent (RSS): Writing ledger to /pool/int/ca24db04-f37b-40b1-83c2-23476bec5a4a/config/.rss-service-plan-v4.json.tmp file = common/src/ledger.rs:199
2502024-09-06T23:35:11.458ZINFOSledAgent (RSS): Writing ledger to /pool/int/358fc814-3f2a-4b1a-aebb-894434d72d01/config/.rss-service-plan-v4.json.tmp file = common/src/ledger.rs:199
2512024-09-06T23:35:11.458ZINFOSledAgent (RSS): Service plan written to storage file = sled-agent/src/rack_setup/plan/service.rs:1001
2522024-09-06T23:35:11.490ZINFOSledAgent (RSS): attempting to set up sled's storage: OmicronPhysicalDisksConfig { generation: Generation(1), disks: [OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, id: 8cc2d138-595a-470d-8336-fb16352cdcc3, pool_id: 652da97e-d553-4457-bc42-d10c0c455967 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, id: cd406e68-4617-4229-ad44-6746e28661aa, pool_id: 1bae97fb-c4f0-4d17-b41c-409574fb9e6e (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, id: 0b5c1f3d-cdb5-4ce0-98c2-820ee6de7412, pool_id: eae5bedf-a363-4cf3-b83a-efcb44445301 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, id: a6b2a0db-152f-47fa-bbff-bf8179871819, pool_id: ab9a060c-a3d5-48de-ba57-beaa10d18520 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, id: 645f632b-a6bd-43af-922c-f8d667e44304, pool_id: fd8d5a1e-9723-41c1-aa2b-460a6959d6c7 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, id: 0f13d27d-43aa-4064-b350-aeba4d67f214, pool_id: b79af253-dd33-4fe7-98c1-790c50e27394 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, id: 3df52921-72ca-47be-8012-5f2c6352bcaf, pool_id: cc3f0ccd-963c-4d0c-843d-e8810b230566 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, id: fe09c378-8877-429e-95dd-849e932a8e74, pool_id: f8e03f3f-0403-4c51-926a-d5ce11d8408e (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, id: a55e9cf7-e6e7-4d43-a59c-5ede4471b7d8, pool_id: 907237be-faec-4233-8c3e-1918ec810764 (zpool) }] } file = sled-agent/src/rack_setup/service.rs:375 sled_address = [fd00:1122:3344:101::1]:12345
2532024-09-06T23:35:11.490ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:47661
2542024-09-06T23:35:11.491ZINFOSledAgent: physical disks ensure file = sled-agent/src/sled_agent.rs:858 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c
2552024-09-06T23:35:11.491ZINFOSledAgent (StorageManager): Received OmicronPhysicalDisksEnsure { config: OmicronPhysicalDisksConfig { generation: Generation(1), disks: [OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, id: 8cc2d138-595a-470d-8336-fb16352cdcc3, pool_id: 652da97e-d553-4457-bc42-d10c0c455967 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, id: cd406e68-4617-4229-ad44-6746e28661aa, pool_id: 1bae97fb-c4f0-4d17-b41c-409574fb9e6e (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, id: 0b5c1f3d-cdb5-4ce0-98c2-820ee6de7412, pool_id: eae5bedf-a363-4cf3-b83a-efcb44445301 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, id: a6b2a0db-152f-47fa-bbff-bf8179871819, pool_id: ab9a060c-a3d5-48de-ba57-beaa10d18520 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, id: 645f632b-a6bd-43af-922c-f8d667e44304, pool_id: fd8d5a1e-9723-41c1-aa2b-460a6959d6c7 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, id: 0f13d27d-43aa-4064-b350-aeba4d67f214, pool_id: b79af253-dd33-4fe7-98c1-790c50e27394 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, id: 3df52921-72ca-47be-8012-5f2c6352bcaf, pool_id: cc3f0ccd-963c-4d0c-843d-e8810b230566 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, id: fe09c378-8877-429e-95dd-849e932a8e74, pool_id: f8e03f3f-0403-4c51-926a-d5ce11d8408e (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, id: a55e9cf7-e6e7-4d43-a59c-5ede4471b7d8, pool_id: 907237be-faec-4233-8c3e-1918ec810764 (zpool) }] }, tx: ... } file = sled-storage/src/manager.rs:451
2562024-09-06T23:35:11.491ZINFOSledAgent (StorageManager): No ledger in /pool/int/ca24db04-f37b-40b1-83c2-23476bec5a4a/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = omicron_physical_disks_ensure
2572024-09-06T23:35:11.491ZINFOSledAgent (StorageManager): No ledger in /pool/int/358fc814-3f2a-4b1a-aebb-894434d72d01/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = omicron_physical_disks_ensure
2582024-09-06T23:35:11.491ZINFOSledAgent (StorageManager): No previously-stored 'requested disks', creating new ledger file = sled-storage/src/manager.rs:866 request = omicron_physical_disks_ensure
2592024-09-06T23:35:11.491ZINFOSledAgent (StorageResources): Synchronizing disk managment file = sled-storage/src/resources.rs:302
2602024-09-06T23:35:11.491ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:342
2612024-09-06T23:35:11.491ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
2622024-09-06T23:35:11.491ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } path = /scratch/u2_0.vdev
2632024-09-06T23:35:11.630ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_0.vdev file = sled-hardware/src/disk.rs:360
2642024-09-06T23:35:11.630ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = 652da97e-d553-4457-bc42-d10c0c455967 (zpool)
2652024-09-06T23:35:11.845ZINFOSledAgent (VmmReservoirManager): Finished setting reservoir size to 49090 MiB bytes file = sled-agent/src/vmm_reservoir.rs:279 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c
2662024-09-06T23:35:11.930ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/55267d89-b780-4916-86a8-64533b7bfd1c): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/probe_manager.rs:230 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c
2672024-09-06T23:35:11.939ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:185 zpool = ZpoolName { id: 652da97e-d553-4457-bc42-d10c0c455967 (zpool), kind: External }
2682024-09-06T23:35:11.947ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:229
2692024-09-06T23:35:11.947ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:231
2702024-09-06T23:35:11.947ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:235
2712024-09-06T23:35:11.947ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:237
2722024-09-06T23:35:11.947ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key")
2732024-09-06T23:35:11.947ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key")
2742024-09-06T23:35:11.947ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:249
2752024-09-06T23:35:11.983ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
2762024-09-06T23:35:11.991ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_652da97e-d553-4457-bc42-d10c0c455967/crypt/zone file = sled-storage/src/dataset.rs:292
2772024-09-06T23:35:12.167ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_652da97e-d553-4457-bc42-d10c0c455967 file = sled-storage/src/dataset.rs:378
2782024-09-06T23:35:12.175ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:440
2792024-09-06T23:35:12.175ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:357
2802024-09-06T23:35:12.175ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:342
2812024-09-06T23:35:12.175ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
2822024-09-06T23:35:12.175ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } path = /scratch/u2_1.vdev
2832024-09-06T23:35:12.185ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_1.vdev file = sled-hardware/src/disk.rs:360
2842024-09-06T23:35:12.185ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = 1bae97fb-c4f0-4d17-b41c-409574fb9e6e (zpool)
2852024-09-06T23:35:12.276ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:185 zpool = ZpoolName { id: 1bae97fb-c4f0-4d17-b41c-409574fb9e6e (zpool), kind: External }
2862024-09-06T23:35:12.284ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:229
2872024-09-06T23:35:12.284ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:231
2882024-09-06T23:35:12.284ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:235
2892024-09-06T23:35:12.284ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:237
2902024-09-06T23:35:12.284ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key")
2912024-09-06T23:35:12.284ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key")
2922024-09-06T23:35:12.284ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:249
2932024-09-06T23:35:12.319ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
2942024-09-06T23:35:12.327ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_1bae97fb-c4f0-4d17-b41c-409574fb9e6e/crypt/zone file = sled-storage/src/dataset.rs:292
2952024-09-06T23:35:12.502ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_1bae97fb-c4f0-4d17-b41c-409574fb9e6e file = sled-storage/src/dataset.rs:378
2962024-09-06T23:35:12.510ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:440
2972024-09-06T23:35:12.510ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:357
2982024-09-06T23:35:12.510ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:342
2992024-09-06T23:35:12.510ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3002024-09-06T23:35:12.510ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } path = /scratch/u2_2.vdev
3012024-09-06T23:35:12.519ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_2.vdev file = sled-hardware/src/disk.rs:360
3022024-09-06T23:35:12.519ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = eae5bedf-a363-4cf3-b83a-efcb44445301 (zpool)
3032024-09-06T23:35:12.614ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/dataset.rs:185 zpool = ZpoolName { id: eae5bedf-a363-4cf3-b83a-efcb44445301 (zpool), kind: External }
3042024-09-06T23:35:12.621ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/dataset.rs:229
3052024-09-06T23:35:12.621ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:231
3062024-09-06T23:35:12.621ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:235
3072024-09-06T23:35:12.621ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:237
3082024-09-06T23:35:12.621ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key")
3092024-09-06T23:35:12.621ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key")
3102024-09-06T23:35:12.621ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:249
3112024-09-06T23:35:12.656ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3122024-09-06T23:35:12.664ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_eae5bedf-a363-4cf3-b83a-efcb44445301/crypt/zone file = sled-storage/src/dataset.rs:292
3132024-09-06T23:35:12.841ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_eae5bedf-a363-4cf3-b83a-efcb44445301 file = sled-storage/src/dataset.rs:378
3142024-09-06T23:35:12.849ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:440
3152024-09-06T23:35:12.849ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:357
3162024-09-06T23:35:12.849ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:342
3172024-09-06T23:35:12.849ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3182024-09-06T23:35:12.849ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } path = /scratch/u2_3.vdev
3192024-09-06T23:35:12.859ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_3.vdev file = sled-hardware/src/disk.rs:360
3202024-09-06T23:35:12.860ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = ab9a060c-a3d5-48de-ba57-beaa10d18520 (zpool)
3212024-09-06T23:35:12.953ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/dataset.rs:185 zpool = ZpoolName { id: ab9a060c-a3d5-48de-ba57-beaa10d18520 (zpool), kind: External }
3222024-09-06T23:35:12.962ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/dataset.rs:229
3232024-09-06T23:35:12.962ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:231
3242024-09-06T23:35:12.962ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:235
3252024-09-06T23:35:12.962ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:237
3262024-09-06T23:35:12.962ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key")
3272024-09-06T23:35:12.963ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key")
3282024-09-06T23:35:12.963ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:249
3292024-09-06T23:35:12.967ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/55267d89-b780-4916-86a8-64533b7bfd1c): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/probe_manager.rs:230 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c
3302024-09-06T23:35:12.998ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3312024-09-06T23:35:13.006ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_ab9a060c-a3d5-48de-ba57-beaa10d18520/crypt/zone file = sled-storage/src/dataset.rs:292
3322024-09-06T23:35:13.187ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_ab9a060c-a3d5-48de-ba57-beaa10d18520 file = sled-storage/src/dataset.rs:378
3332024-09-06T23:35:13.195ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:440
3342024-09-06T23:35:13.195ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:357
3352024-09-06T23:35:13.195ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:342
3362024-09-06T23:35:13.195ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3372024-09-06T23:35:13.195ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } path = /scratch/u2_4.vdev
3382024-09-06T23:35:13.206ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_4.vdev file = sled-hardware/src/disk.rs:360
3392024-09-06T23:35:13.206ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = fd8d5a1e-9723-41c1-aa2b-460a6959d6c7 (zpool)
3402024-09-06T23:35:13.300ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/dataset.rs:185 zpool = ZpoolName { id: fd8d5a1e-9723-41c1-aa2b-460a6959d6c7 (zpool), kind: External }
3412024-09-06T23:35:13.307ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/dataset.rs:229
3422024-09-06T23:35:13.307ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:231
3432024-09-06T23:35:13.307ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:235
3442024-09-06T23:35:13.307ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:237
3452024-09-06T23:35:13.307ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key")
3462024-09-06T23:35:13.308ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key")
3472024-09-06T23:35:13.308ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:249
3482024-09-06T23:35:13.344ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3492024-09-06T23:35:13.352ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_fd8d5a1e-9723-41c1-aa2b-460a6959d6c7/crypt/zone file = sled-storage/src/dataset.rs:292
3502024-09-06T23:35:13.524ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_fd8d5a1e-9723-41c1-aa2b-460a6959d6c7 file = sled-storage/src/dataset.rs:378
3512024-09-06T23:35:13.532ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:440
3522024-09-06T23:35:13.532ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:357
3532024-09-06T23:35:13.532ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:342
3542024-09-06T23:35:13.532ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3552024-09-06T23:35:13.532ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } path = /scratch/u2_5.vdev
3562024-09-06T23:35:13.542ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_5.vdev file = sled-hardware/src/disk.rs:360
3572024-09-06T23:35:13.542ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = b79af253-dd33-4fe7-98c1-790c50e27394 (zpool)
3582024-09-06T23:35:13.639ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/dataset.rs:185 zpool = ZpoolName { id: b79af253-dd33-4fe7-98c1-790c50e27394 (zpool), kind: External }
3592024-09-06T23:35:13.647ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/dataset.rs:229
3602024-09-06T23:35:13.647ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:231
3612024-09-06T23:35:13.647ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:235
3622024-09-06T23:35:13.647ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:237
3632024-09-06T23:35:13.647ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key")
3642024-09-06T23:35:13.647ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key")
3652024-09-06T23:35:13.647ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:249
3662024-09-06T23:35:13.683ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3672024-09-06T23:35:13.690ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_b79af253-dd33-4fe7-98c1-790c50e27394/crypt/zone file = sled-storage/src/dataset.rs:292
3682024-09-06T23:35:13.858ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 17.885310933s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c
3692024-09-06T23:35:13.868ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_b79af253-dd33-4fe7-98c1-790c50e27394 file = sled-storage/src/dataset.rs:378
3702024-09-06T23:35:13.876ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:440
3712024-09-06T23:35:13.876ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:357
3722024-09-06T23:35:13.876ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:342
3732024-09-06T23:35:13.876ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3742024-09-06T23:35:13.876ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } path = /scratch/u2_6.vdev
3752024-09-06T23:35:13.885ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_6.vdev file = sled-hardware/src/disk.rs:360
3762024-09-06T23:35:13.885ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = cc3f0ccd-963c-4d0c-843d-e8810b230566 (zpool)
3772024-09-06T23:35:13.982ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/dataset.rs:185 zpool = ZpoolName { id: cc3f0ccd-963c-4d0c-843d-e8810b230566 (zpool), kind: External }
3782024-09-06T23:35:13.990ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/dataset.rs:229
3792024-09-06T23:35:13.990ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:231
3802024-09-06T23:35:13.990ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:235
3812024-09-06T23:35:13.990ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:237
3822024-09-06T23:35:13.990ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key")
3832024-09-06T23:35:13.990ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key")
3842024-09-06T23:35:13.990ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:249
3852024-09-06T23:35:13.994ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/55267d89-b780-4916-86a8-64533b7bfd1c): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/probe_manager.rs:230 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c
3862024-09-06T23:35:14.026ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3872024-09-06T23:35:14.034ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_cc3f0ccd-963c-4d0c-843d-e8810b230566/crypt/zone file = sled-storage/src/dataset.rs:292
3882024-09-06T23:35:14.211ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_cc3f0ccd-963c-4d0c-843d-e8810b230566 file = sled-storage/src/dataset.rs:378
3892024-09-06T23:35:14.219ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:440
3902024-09-06T23:35:14.219ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:357
3912024-09-06T23:35:14.219ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:342
3922024-09-06T23:35:14.219ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3932024-09-06T23:35:14.219ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } path = /scratch/u2_7.vdev
3942024-09-06T23:35:14.229ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_7.vdev file = sled-hardware/src/disk.rs:360
3952024-09-06T23:35:14.229ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = f8e03f3f-0403-4c51-926a-d5ce11d8408e (zpool)
3962024-09-06T23:35:14.327ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/dataset.rs:185 zpool = ZpoolName { id: f8e03f3f-0403-4c51-926a-d5ce11d8408e (zpool), kind: External }
3972024-09-06T23:35:14.336ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/dataset.rs:229
3982024-09-06T23:35:14.336ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:231
3992024-09-06T23:35:14.336ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:235
4002024-09-06T23:35:14.336ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:237
4012024-09-06T23:35:14.336ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key")
4022024-09-06T23:35:14.337ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key")
4032024-09-06T23:35:14.337ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:249
4042024-09-06T23:35:14.375ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
4052024-09-06T23:35:14.384ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_f8e03f3f-0403-4c51-926a-d5ce11d8408e/crypt/zone file = sled-storage/src/dataset.rs:292
4062024-09-06T23:35:14.563ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_f8e03f3f-0403-4c51-926a-d5ce11d8408e file = sled-storage/src/dataset.rs:378
4072024-09-06T23:35:14.571ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:440
4082024-09-06T23:35:14.571ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:357
4092024-09-06T23:35:14.571ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:342
4102024-09-06T23:35:14.571ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
4112024-09-06T23:35:14.571ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } path = /scratch/u2_8.vdev
4122024-09-06T23:35:14.581ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_8.vdev file = sled-hardware/src/disk.rs:360
4132024-09-06T23:35:14.581ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = 907237be-faec-4233-8c3e-1918ec810764 (zpool)
4142024-09-06T23:35:14.686ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/dataset.rs:185 zpool = ZpoolName { id: 907237be-faec-4233-8c3e-1918ec810764 (zpool), kind: External }
4152024-09-06T23:35:14.694ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/dataset.rs:229
4162024-09-06T23:35:14.694ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:231
4172024-09-06T23:35:14.694ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:235
4182024-09-06T23:35:14.694ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:237
4192024-09-06T23:35:14.694ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key")
4202024-09-06T23:35:14.695ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key")
4212024-09-06T23:35:14.695ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:249
4222024-09-06T23:35:14.730ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
4232024-09-06T23:35:14.739ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_907237be-faec-4233-8c3e-1918ec810764/crypt/zone file = sled-storage/src/dataset.rs:292
4242024-09-06T23:35:14.912ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_907237be-faec-4233-8c3e-1918ec810764 file = sled-storage/src/dataset.rs:378
4252024-09-06T23:35:14.920ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:440
4262024-09-06T23:35:14.920ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:357
4272024-09-06T23:35:14.920ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/ca24db04-f37b-40b1-83c2-23476bec5a4a/config/.omicron-physical-disks.json.tmp file = common/src/ledger.rs:199 request = omicron_physical_disks_ensure
4282024-09-06T23:35:14.920ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: ca24db04-f37b-40b1-83c2-23476bec5a4a (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 358fc814-3f2a-4b1a-aebb-894434d72d01 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 652da97e-d553-4457-bc42-d10c0c455967 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 1bae97fb-c4f0-4d17-b41c-409574fb9e6e (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: eae5bedf-a363-4cf3-b83a-efcb44445301 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: ab9a060c-a3d5-48de-ba57-beaa10d18520 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: fd8d5a1e-9723-41c1-aa2b-460a6959d6c7 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: b79af253-dd33-4fe7-98c1-790c50e27394 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: cc3f0ccd-963c-4d0c-843d-e8810b230566 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: f8e03f3f-0403-4c51-926a-d5ce11d8408e (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 907237be-faec-4233-8c3e-1918ec810764 (zpool), kind: External } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
4292024-09-06T23:35:14.920ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
4302024-09-06T23:35:14.920ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
4312024-09-06T23:35:14.921ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/358fc814-3f2a-4b1a-aebb-894434d72d01/config/.omicron-physical-disks.json.tmp file = common/src/ledger.rs:199 request = omicron_physical_disks_ensure
4322024-09-06T23:35:14.921ZINFOSledAgent: physical disks ensure: Updated storage file = sled-agent/src/sled_agent.rs:862 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c
4332024-09-06T23:35:14.921ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
4342024-09-06T23:35:14.921ZINFOSledAgent: physical disks ensure: Propagating new generation of disks file = sled-agent/src/sled_agent.rs:888 generation = Generation(1) sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c
4352024-09-06T23:35:14.921ZINFOSledAgent: physical disks ensure: Updated storage monitor file = sled-agent/src/sled_agent.rs:893 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c
4362024-09-06T23:35:14.921ZINFOSledAgent: physical disks ensure: Updated zone bundler file = sled-agent/src/sled_agent.rs:898 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c
4372024-09-06T23:35:14.921ZINFOSledAgent (ProbeManager): use_only_these_disks: Processing new request file = sled-agent/src/probe_manager.rs:121 gen = Generation(1) sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c
4382024-09-06T23:35:14.921ZINFOSledAgent: physical disks ensure: Updated probes file = sled-agent/src/sled_agent.rs:904 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c
4392024-09-06T23:35:14.921ZINFOSledAgent (InstanceManager): use_only_these_disks: Processing new request file = sled-agent/src/instance_manager.rs:803 gen = Generation(1)
4402024-09-06T23:35:14.921ZINFOSledAgent: physical disks ensure: Updated instances file = sled-agent/src/sled_agent.rs:909 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c
4412024-09-06T23:35:14.921ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3430475 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:47661 req_id = 5287df88-3cc1-4e91-ad5e-f7e15668ba05 response_code = 200 uri = /omicron-physical-disks
4422024-09-06T23:35:14.953ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(2), zones: [OmicronZoneConfig { id: 26e8255e-2d98-41fa-b877-69c2893c27d0, underlay_address: fd00:1122:3344:1::1, filesystem_pool: Some(ZpoolName { id: 652da97e-d553-4457-bc42-d10c0c455967 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 652da97e-d553-4457-bc42-d10c0c455967 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: fcc84a23-e14a-4383-9764-158f0d5fc6c9, underlay_address: fd00:1122:3344:2::1, filesystem_pool: Some(ZpoolName { id: 1bae97fb-c4f0-4d17-b41c-409574fb9e6e (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 1bae97fb-c4f0-4d17-b41c-409574fb9e6e (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: a717f03d-27a8-4fde-9656-972b10665736, underlay_address: fd00:1122:3344:3::1, filesystem_pool: Some(ZpoolName { id: eae5bedf-a363-4cf3-b83a-efcb44445301 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: eae5bedf-a363-4cf3-b83a-efcb44445301 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }] } file = sled-agent/src/rack_setup/service.rs:458 sled_address = [fd00:1122:3344:101::1]:12345
4432024-09-06T23:35:14.953ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:58261
4442024-09-06T23:35:14.954ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 26e8255e-2d98-41fa-b877-69c2893c27d0, dataset_name: DatasetName { pool_name: ZpoolName { id: 652da97e-d553-4457-bc42-d10c0c455967 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
4452024-09-06T23:35:14.954ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 26e8255e-2d98-41fa-b877-69c2893c27d0, dataset_name: DatasetName { pool_name: ZpoolName { id: 652da97e-d553-4457-bc42-d10c0c455967 (zpool), kind: External }, kind: InternalDns }, responder: ... }
4462024-09-06T23:35:14.997ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/55267d89-b780-4916-86a8-64533b7bfd1c): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/probe_manager.rs:230 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c
4472024-09-06T23:35:15.003ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: fcc84a23-e14a-4383-9764-158f0d5fc6c9, dataset_name: DatasetName { pool_name: ZpoolName { id: 1bae97fb-c4f0-4d17-b41c-409574fb9e6e (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
4482024-09-06T23:35:15.003ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: fcc84a23-e14a-4383-9764-158f0d5fc6c9, dataset_name: DatasetName { pool_name: ZpoolName { id: 1bae97fb-c4f0-4d17-b41c-409574fb9e6e (zpool), kind: External }, kind: InternalDns }, responder: ... }
4492024-09-06T23:35:15.056ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: a717f03d-27a8-4fde-9656-972b10665736, dataset_name: DatasetName { pool_name: ZpoolName { id: eae5bedf-a363-4cf3-b83a-efcb44445301 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
4502024-09-06T23:35:15.056ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: a717f03d-27a8-4fde-9656-972b10665736, dataset_name: DatasetName { pool_name: ZpoolName { id: eae5bedf-a363-4cf3-b83a-efcb44445301 (zpool), kind: External }, kind: InternalDns }, responder: ... }
4512024-09-06T23:35:15.107ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
4522024-09-06T23:35:15.107ZINFOSledAgent (ServiceManager): No ledger in /pool/int/ca24db04-f37b-40b1-83c2-23476bec5a4a/config/omicron-zones.json file = common/src/ledger.rs:188
4532024-09-06T23:35:15.107ZINFOSledAgent (ServiceManager): No ledger in /pool/int/358fc814-3f2a-4b1a-aebb-894434d72d01/config/omicron-zones.json file = common/src/ledger.rs:188
4542024-09-06T23:35:15.107ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
4552024-09-06T23:35:15.150ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
4562024-09-06T23:35:15.150ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
4572024-09-06T23:35:15.150ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
4582024-09-06T23:35:15.201ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_a717f03d-27a8-4fde-9656-972b10665736 file = illumos-utils/src/zone.rs:316
4592024-09-06T23:35:15.205ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_26e8255e-2d98-41fa-b877-69c2893c27d0 file = illumos-utils/src/zone.rs:316
4602024-09-06T23:35:15.209ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_fcc84a23-e14a-4383-9764-158f0d5fc6c9 file = illumos-utils/src/zone.rs:316
4612024-09-06T23:35:15.222ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_a717f03d-27a8-4fde-9656-972b10665736 file = illumos-utils/src/zone.rs:354
4622024-09-06T23:35:15.227ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_26e8255e-2d98-41fa-b877-69c2893c27d0 file = illumos-utils/src/zone.rs:354
4632024-09-06T23:35:15.230ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_fcc84a23-e14a-4383-9764-158f0d5fc6c9 file = illumos-utils/src/zone.rs:354
4642024-09-06T23:35:16.000ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/55267d89-b780-4916-86a8-64533b7bfd1c): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/probe_manager.rs:230 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c
4652024-09-06T23:35:16.770ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_fcc84a23-e14a-4383-9764-158f0d5fc6c9: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService3'/> <propval type="astring" name="gateway" value='fd00:1122:3344:2::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:2::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:2::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:2::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
4662024-09-06T23:35:16.770ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:2::, width: 64 }
4672024-09-06T23:35:17.248ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_a717f03d-27a8-4fde-9656-972b10665736: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService1'/> <propval type="astring" name="gateway" value='fd00:1122:3344:3::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:3::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:3::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:3::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
4682024-09-06T23:35:17.725ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_26e8255e-2d98-41fa-b877-69c2893c27d0: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService2'/> <propval type="astring" name="gateway" value='fd00:1122:3344:1::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:1::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:1::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:1::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
4692024-09-06T23:35:17.725ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:3::, width: 64 }
4702024-09-06T23:35:17.725ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_fcc84a23-e14a-4383-9764-158f0d5fc6c9 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_fcc84a23-e14a-4383-9764-158f0d5fc6c9
4712024-09-06T23:35:17.729ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:1::, width: 64 }
4722024-09-06T23:35:17.729ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_a717f03d-27a8-4fde-9656-972b10665736 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_a717f03d-27a8-4fde-9656-972b10665736
4732024-09-06T23:35:17.733ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_26e8255e-2d98-41fa-b877-69c2893c27d0 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_26e8255e-2d98-41fa-b877-69c2893c27d0
4742024-09-06T23:35:17.853ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_fcc84a23-e14a-4383-9764-158f0d5fc6c9") failed: Property not found. retry in 72.23211ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_fcc84a23-e14a-4383-9764-158f0d5fc6c9
4752024-09-06T23:35:17.862ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_a717f03d-27a8-4fde-9656-972b10665736") failed: Property not found. retry in 37.807718ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_a717f03d-27a8-4fde-9656-972b10665736
4762024-09-06T23:35:17.869ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_26e8255e-2d98-41fa-b877-69c2893c27d0") failed: Property not found. retry in 60.653331ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_26e8255e-2d98-41fa-b877-69c2893c27d0
4772024-09-06T23:35:17.907ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_a717f03d-27a8-4fde-9656-972b10665736") failed: Property not found. retry in 119.909622ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_a717f03d-27a8-4fde-9656-972b10665736
4782024-09-06T23:35:18.113ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_fcc84a23-e14a-4383-9764-158f0d5fc6c9") failed: Property not found. retry in 66.317657ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_fcc84a23-e14a-4383-9764-158f0d5fc6c9
4792024-09-06T23:35:18.125ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_a717f03d-27a8-4fde-9656-972b10665736") failed: Property not found. retry in 237.709528ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_a717f03d-27a8-4fde-9656-972b10665736
4802024-09-06T23:35:18.146ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_26e8255e-2d98-41fa-b877-69c2893c27d0") failed: Property not found. retry in 54.738851ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_26e8255e-2d98-41fa-b877-69c2893c27d0
4812024-09-06T23:35:18.174ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_fcc84a23-e14a-4383-9764-158f0d5fc6c9") failed: Property not found. retry in 222.520859ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_fcc84a23-e14a-4383-9764-158f0d5fc6c9
4822024-09-06T23:35:18.218ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_26e8255e-2d98-41fa-b877-69c2893c27d0") failed: Property not found. retry in 169.232879ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_26e8255e-2d98-41fa-b877-69c2893c27d0
4832024-09-06T23:35:18.389ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_a717f03d-27a8-4fde-9656-972b10665736") failed: Property not found. retry in 585.227906ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_a717f03d-27a8-4fde-9656-972b10665736
4842024-09-06T23:35:18.427ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_26e8255e-2d98-41fa-b877-69c2893c27d0") failed: Property not found. retry in 498.251144ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_26e8255e-2d98-41fa-b877-69c2893c27d0
4852024-09-06T23:35:18.439ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_fcc84a23-e14a-4383-9764-158f0d5fc6c9") failed: Property not found. retry in 431.338908ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_fcc84a23-e14a-4383-9764-158f0d5fc6c9
4862024-09-06T23:35:18.881ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_fcc84a23-e14a-4383-9764-158f0d5fc6c9") failed: Property not found. retry in 1.037721265s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_fcc84a23-e14a-4383-9764-158f0d5fc6c9
4872024-09-06T23:35:18.921ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_26e8255e-2d98-41fa-b877-69c2893c27d0") failed: Property not found. retry in 1.102483882s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_26e8255e-2d98-41fa-b877-69c2893c27d0
4882024-09-06T23:35:18.982ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_a717f03d-27a8-4fde-9656-972b10665736") failed: Property not found. retry in 854.158834ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_a717f03d-27a8-4fde-9656-972b10665736
4892024-09-06T23:35:19.854ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_a717f03d-27a8-4fde-9656-972b10665736") failed: Property not found. retry in 975.475805ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_a717f03d-27a8-4fde-9656-972b10665736
4902024-09-06T23:35:19.930ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_fcc84a23-e14a-4383-9764-158f0d5fc6c9") failed: Property not found. retry in 1.185791018s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_fcc84a23-e14a-4383-9764-158f0d5fc6c9
4912024-09-06T23:35:20.034ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_26e8255e-2d98-41fa-b877-69c2893c27d0") failed: Property not found. retry in 817.635775ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_26e8255e-2d98-41fa-b877-69c2893c27d0
4922024-09-06T23:35:20.310ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::2 exists - OK file = sled-agent/src/services.rs:4187
4932024-09-06T23:35:20.310ZINFOSledAgent (ServiceManager): Ensuring there is a default route file = sled-agent/src/services.rs:4197 gateway = fd00:1122:3344:101::1
4942024-09-06T23:35:20.316ZINFOSledAgent (ServiceManager): configuring dendrite service file = sled-agent/src/services.rs:4276
4952024-09-06T23:35:20.445ZINFOSledAgent (ServiceManager): refreshed dendrite service with new configuration file = sled-agent/src/services.rs:4324
4962024-09-06T23:35:20.445ZINFOSledAgent (ServiceManager): configuring lldp service file = sled-agent/src/services.rs:4351
4972024-09-06T23:35:20.485ZINFOSledAgent (ServiceManager): refreshed lldpd service with new configuration file = sled-agent/src/services.rs:4364
4982024-09-06T23:35:20.485ZINFOSledAgent (ServiceManager): configuring MGS service file = sled-agent/src/services.rs:4228
4992024-09-06T23:35:20.536ZINFOSledAgent (ServiceManager): refreshed MGS service with new configuration file = sled-agent/src/services.rs:4270
5002024-09-06T23:35:20.536ZINFOSledAgent (ServiceManager): configuring wicketd file = sled-agent/src/services.rs:4331 rack_subnet = fd00:1122:3344::
5012024-09-06T23:35:20.558ZINFOSledAgent (ServiceManager): refreshed wicketd service with new configuration file = sled-agent/src/services.rs:4342
5022024-09-06T23:35:20.558ZINFOSledAgent (ServiceManager): configuring mgd service file = sled-agent/src/services.rs:4384
5032024-09-06T23:35:20.648ZINFOSledAgent (ServiceManager): refreshed mgd service with new configuration file = sled-agent/src/services.rs:4416
5042024-09-06T23:35:20.648ZINFOSledAgent (ServiceManager): configuring mg-ddm service file = sled-agent/src/services.rs:4422
5052024-09-06T23:35:20.760ZINFOSledAgent (ServiceManager): refreshed mg-ddm service with new configuration file = sled-agent/src/services.rs:4463
5062024-09-06T23:35:20.760ZINFOSledAgent (ServiceManager): Determining physical location of our switch zone at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:366
5072024-09-06T23:35:20.793ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/55267d89-b780-4916-86a8-64533b7bfd1c): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/nexus.rs:571 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c
5082024-09-06T23:35:20.794ZWARNSledAgent (ServiceManager): Failed to get switch ID from MGS (retrying in 35.680499ms) error = Communication Error: error sending request for url (http://[fd00:1122:3344:101::2]:12225/local/switch-id): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/bootstrap/early_networking.rs:385
5092024-09-06T23:35:20.830ZWARNSledAgent (ServiceManager): Failed to get switch ID from MGS (retrying in 77.202807ms) error = Communication Error: error sending request for url (http://[fd00:1122:3344:101::2]:12225/local/switch-id): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/bootstrap/early_networking.rs:385
5102024-09-06T23:35:20.839ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_a717f03d-27a8-4fde-9656-972b10665736") failed: Property not found. retry in 1.090049167s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_a717f03d-27a8-4fde-9656-972b10665736
5112024-09-06T23:35:20.859ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_26e8255e-2d98-41fa-b877-69c2893c27d0") failed: Property not found. retry in 654.571529ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_26e8255e-2d98-41fa-b877-69c2893c27d0
5122024-09-06T23:35:20.908ZWARNSledAgent (ServiceManager): Failed to get switch ID from MGS (retrying in 158.942024ms) error = Communication Error: error sending request for url (http://[fd00:1122:3344:101::2]:12225/local/switch-id): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/bootstrap/early_networking.rs:385
5132024-09-06T23:35:21.068ZWARNSledAgent (ServiceManager): Failed to get switch ID from MGS (retrying in 340.220558ms) error = Communication Error: error sending request for url (http://[fd00:1122:3344:101::2]:12225/local/switch-id): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/bootstrap/early_networking.rs:385
5142024-09-06T23:35:21.124ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_fcc84a23-e14a-4383-9764-158f0d5fc6c9") failed: Property not found. retry in 983.328411ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_fcc84a23-e14a-4383-9764-158f0d5fc6c9
5152024-09-06T23:35:21.306ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::2]:64421
5162024-09-06T23:35:21.306ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 98 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::2]:64421 req_id = 2b231f9c-0121-403d-b330-445ad99ee435 response_code = 200 uri = /sled-identifiers
5172024-09-06T23:35:21.409ZINFOSledAgent (ServiceManager): Initializing 1 Uplinks on Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:414
5182024-09-06T23:35:21.442ZINFOSledAgent (ServiceManager): Building Port Configuration file = sled-agent/src/bootstrap/early_networking.rs:676
5192024-09-06T23:35:21.442ZINFOSledAgent (ServiceManager): Checking dendrite uptime file = sled-agent/src/bootstrap/early_networking.rs:714
5202024-09-06T23:35:21.442ZINFOSledAgent (ServiceManager): Dendrite online file = sled-agent/src/bootstrap/early_networking.rs:717 uptime = 0
5212024-09-06T23:35:21.442ZINFOSledAgent (ServiceManager): Configuring default uplink on switch config = PortSettings {\n links: {\n "0": LinkSettings {\n addrs: [\n 10.151.2.174,\n ],\n params: LinkCreate {\n autoneg: false,\n fec: None,\n kr: false,\n lane: Some(\n LinkId(\n 0,\n ),\n ),\n speed: Speed40G,\n },\n },\n },\n} file = sled-agent/src/bootstrap/early_networking.rs:436
5222024-09-06T23:35:21.480ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4014
5232024-09-06T23:35:21.523ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_26e8255e-2d98-41fa-b877-69c2893c27d0") failed: Property not found. retry in 960.087139ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_26e8255e-2d98-41fa-b877-69c2893c27d0
5242024-09-06T23:35:21.529ZINFOSledAgent (StorageManager): Received DetectedRawDisksChanged { raw_disks: {}, tx: ... } file = sled-storage/src/manager.rs:451
5252024-09-06T23:35:21.529ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
5262024-09-06T23:35:21.529ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
5272024-09-06T23:35:21.529ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
5282024-09-06T23:35:21.529ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
5292024-09-06T23:35:21.529ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
5302024-09-06T23:35:21.529ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
5312024-09-06T23:35:21.529ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
5322024-09-06T23:35:21.529ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
5332024-09-06T23:35:21.529ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
5342024-09-06T23:35:21.529ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
5352024-09-06T23:35:21.529ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
5362024-09-06T23:35:21.529ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
5372024-09-06T23:35:21.529ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
5382024-09-06T23:35:21.529ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
5392024-09-06T23:35:21.529ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
5402024-09-06T23:35:21.529ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
5412024-09-06T23:35:21.529ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
5422024-09-06T23:35:21.529ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
5432024-09-06T23:35:21.529ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
5442024-09-06T23:35:21.529ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
5452024-09-06T23:35:21.529ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
5462024-09-06T23:35:21.529ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
5472024-09-06T23:35:21.939ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_a717f03d-27a8-4fde-9656-972b10665736") failed: Property not found. retry in 1.478000296s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_a717f03d-27a8-4fde-9656-972b10665736
5482024-09-06T23:35:22.119ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_fcc84a23-e14a-4383-9764-158f0d5fc6c9") failed: Property not found. retry in 1.026169722s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_fcc84a23-e14a-4383-9764-158f0d5fc6c9
5492024-09-06T23:35:22.492ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_26e8255e-2d98-41fa-b877-69c2893c27d0") failed: Property not found. retry in 770.05007ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_26e8255e-2d98-41fa-b877-69c2893c27d0
5502024-09-06T23:35:23.437ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3226 zone = oxz_internal_dns_a717f03d-27a8-4fde-9656-972b10665736
5512024-09-06T23:35:23.437ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3226 zone = oxz_internal_dns_26e8255e-2d98-41fa-b877-69c2893c27d0
5522024-09-06T23:35:23.437ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3226 zone = oxz_internal_dns_fcc84a23-e14a-4383-9764-158f0d5fc6c9
5532024-09-06T23:35:23.437ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/ca24db04-f37b-40b1-83c2-23476bec5a4a/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
5542024-09-06T23:35:23.437ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/358fc814-3f2a-4b1a-aebb-894434d72d01/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
5552024-09-06T23:35:23.438ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8484285 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:58261 req_id = be5c94b8-9026-4649-81eb-234a378d3f4e response_code = 204 uri = /omicron-zones
5562024-09-06T23:35:23.439ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:1::1]:5353 file = sled-agent/src/rack_setup/service.rs:575
5572024-09-06T23:35:23.594ZWARNSledAgent (RSS): failed to write DNS configuration (will retry in 78.3582ms) dns_config_addr = [fd00:1122:3344:1::1]:5353 error_message = Communication Error: error sending request for url (http://[fd00:1122:3344:1::1]:5353/config): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:595
5582024-09-06T23:35:23.690ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:2::1]:5353 file = sled-agent/src/rack_setup/service.rs:575
5592024-09-06T23:35:23.737ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:3::1]:5353 file = sled-agent/src/rack_setup/service.rs:575
5602024-09-06T23:35:23.783ZINFOSledAgent (RSS): Configured all DNS servers file = sled-agent/src/rack_setup/service.rs:611
5612024-09-06T23:35:23.783ZINFOSledAgent (RSS): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
5622024-09-06T23:35:23.816ZINFOSledAgent (RSS): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
5632024-09-06T23:35:23.817ZINFOSledAgent (RSS): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
5642024-09-06T23:35:23.817ZINFOSledAgent (RSS): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
5652024-09-06T23:35:23.847ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(3), zones: [OmicronZoneConfig { id: 26e8255e-2d98-41fa-b877-69c2893c27d0, underlay_address: fd00:1122:3344:1::1, filesystem_pool: Some(ZpoolName { id: 652da97e-d553-4457-bc42-d10c0c455967 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 652da97e-d553-4457-bc42-d10c0c455967 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: fcc84a23-e14a-4383-9764-158f0d5fc6c9, underlay_address: fd00:1122:3344:2::1, filesystem_pool: Some(ZpoolName { id: 1bae97fb-c4f0-4d17-b41c-409574fb9e6e (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 1bae97fb-c4f0-4d17-b41c-409574fb9e6e (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: a717f03d-27a8-4fde-9656-972b10665736, underlay_address: fd00:1122:3344:3::1, filesystem_pool: Some(ZpoolName { id: eae5bedf-a363-4cf3-b83a-efcb44445301 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: eae5bedf-a363-4cf3-b83a-efcb44445301 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 563f2b19-0f47-40c0-8fff-c29b65f02314, underlay_address: fd00:1122:3344:101::1b, filesystem_pool: Some(ZpoolName { id: eae5bedf-a363-4cf3-b83a-efcb44445301 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: acd43372-de8f-49b0-a1ec-6f7d2718bb22, kind: Service { id: 563f2b19-0f47-40c0-8fff-c29b65f02314 }, name: Name("ntp-563f2b19-0f47-40c0-8fff-c29b65f02314"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 202])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }] } file = sled-agent/src/rack_setup/service.rs:458 sled_address = [fd00:1122:3344:101::1]:12345
5662024-09-06T23:35:23.847ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:35852
5672024-09-06T23:35:23.847ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 26e8255e-2d98-41fa-b877-69c2893c27d0, dataset_name: DatasetName { pool_name: ZpoolName { id: 652da97e-d553-4457-bc42-d10c0c455967 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
5682024-09-06T23:35:23.847ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 26e8255e-2d98-41fa-b877-69c2893c27d0, dataset_name: DatasetName { pool_name: ZpoolName { id: 652da97e-d553-4457-bc42-d10c0c455967 (zpool), kind: External }, kind: InternalDns }, responder: ... }
5692024-09-06T23:35:23.870ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: fcc84a23-e14a-4383-9764-158f0d5fc6c9, dataset_name: DatasetName { pool_name: ZpoolName { id: 1bae97fb-c4f0-4d17-b41c-409574fb9e6e (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
5702024-09-06T23:35:23.870ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: fcc84a23-e14a-4383-9764-158f0d5fc6c9, dataset_name: DatasetName { pool_name: ZpoolName { id: 1bae97fb-c4f0-4d17-b41c-409574fb9e6e (zpool), kind: External }, kind: InternalDns }, responder: ... }
5712024-09-06T23:35:23.890ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: a717f03d-27a8-4fde-9656-972b10665736, dataset_name: DatasetName { pool_name: ZpoolName { id: eae5bedf-a363-4cf3-b83a-efcb44445301 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
5722024-09-06T23:35:23.890ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: a717f03d-27a8-4fde-9656-972b10665736, dataset_name: DatasetName { pool_name: ZpoolName { id: eae5bedf-a363-4cf3-b83a-efcb44445301 (zpool), kind: External }, kind: InternalDns }, responder: ... }
5732024-09-06T23:35:23.909ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
5742024-09-06T23:35:23.909ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/ca24db04-f37b-40b1-83c2-23476bec5a4a/config/omicron-zones.json file = common/src/ledger.rs:177
5752024-09-06T23:35:23.909ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/358fc814-3f2a-4b1a-aebb-894434d72d01/config/omicron-zones.json file = common/src/ledger.rs:177
5762024-09-06T23:35:23.909ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
5772024-09-06T23:35:23.920ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
5782024-09-06T23:35:23.952ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
5792024-09-06T23:35:23.953ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
5802024-09-06T23:35:23.953ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
5812024-09-06T23:35:23.983ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte0", ip: 172.30.3.5, mac: MacAddr6([168, 64, 37, 255, 222, 202]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.3.1 } }) }
5822024-09-06T23:35:23.983ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1234 zone_type = boundary_ntp
5832024-09-06T23:35:23.984ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.169 file = clients/dpd-client/src/lib.rs:83
5842024-09-06T23:35:23.984ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.169 file = clients/dpd-client/src/lib.rs:90
5852024-09-06T23:35:23.984ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.169 file = clients/dpd-client/src/lib.rs:118
5862024-09-06T23:35:23.984ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
5872024-09-06T23:35:24.010ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_ntp_563f2b19-0f47-40c0-8fff-c29b65f02314 file = illumos-utils/src/zone.rs:316
5882024-09-06T23:35:24.030ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_ntp_563f2b19-0f47-40c0-8fff-c29b65f02314 file = illumos-utils/src/zone.rs:354
5892024-09-06T23:35:24.658ZINFOSledAgent (ServiceManager): Profile for oxz_ntp_563f2b19-0f47-40c0-8fff-c29b65f02314: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService4'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::1b'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/chrony-setup"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="allow" value='fd00:1122:3344:100::/56'/> <propval type="boolean" name="boundary" value='true'/> <propval type="astring" name="boundary_pool" value='boundary-ntp.control-plane.oxide.internal'/> <propval type="astring" name="server" value='0.pool.ntp.org'/> </property_group> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="1.1.1.1" /> <value_node value="9.9.9.9" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/ntp"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte0'/> <propval type="astring" name="gateway" value='172.30.3.1'/> <propval type="astring" name="ip" value='172.30.3.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
5902024-09-06T23:35:24.658ZINFOSledAgent (ServiceManager): Booting oxz_ntp_563f2b19-0f47-40c0-8fff-c29b65f02314 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_ntp_563f2b19-0f47-40c0-8fff-c29b65f02314
5912024-09-06T23:35:24.795ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_563f2b19-0f47-40c0-8fff-c29b65f02314") failed: Property not found. retry in 34.040202ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_563f2b19-0f47-40c0-8fff-c29b65f02314
5922024-09-06T23:35:24.837ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_563f2b19-0f47-40c0-8fff-c29b65f02314") failed: Property not found. retry in 88.931315ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_563f2b19-0f47-40c0-8fff-c29b65f02314
5932024-09-06T23:35:24.938ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_563f2b19-0f47-40c0-8fff-c29b65f02314") failed: Property not found. retry in 262.189222ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_563f2b19-0f47-40c0-8fff-c29b65f02314
5942024-09-06T23:35:25.213ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_563f2b19-0f47-40c0-8fff-c29b65f02314") failed: Property not found. retry in 265.709866ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_563f2b19-0f47-40c0-8fff-c29b65f02314
5952024-09-06T23:35:25.487ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_563f2b19-0f47-40c0-8fff-c29b65f02314") failed: Property not found. retry in 494.429314ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_563f2b19-0f47-40c0-8fff-c29b65f02314
5962024-09-06T23:35:25.992ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_563f2b19-0f47-40c0-8fff-c29b65f02314") failed: Property not found. retry in 638.42848ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_563f2b19-0f47-40c0-8fff-c29b65f02314
5972024-09-06T23:35:26.641ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_563f2b19-0f47-40c0-8fff-c29b65f02314") failed: Property not found. retry in 827.876089ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_563f2b19-0f47-40c0-8fff-c29b65f02314
5982024-09-06T23:35:27.478ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_563f2b19-0f47-40c0-8fff-c29b65f02314") failed: Property not found. retry in 821.45342ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_563f2b19-0f47-40c0-8fff-c29b65f02314
5992024-09-06T23:35:28.308ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_563f2b19-0f47-40c0-8fff-c29b65f02314") failed: Property not found. retry in 573.353422ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_563f2b19-0f47-40c0-8fff-c29b65f02314
6002024-09-06T23:35:28.891ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_563f2b19-0f47-40c0-8fff-c29b65f02314") failed: Property not found. retry in 530.61808ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_563f2b19-0f47-40c0-8fff-c29b65f02314
6012024-09-06T23:35:29.431ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_563f2b19-0f47-40c0-8fff-c29b65f02314") failed: Property not found. retry in 623.130333ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_563f2b19-0f47-40c0-8fff-c29b65f02314
6022024-09-06T23:35:30.075ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3226 zone = oxz_ntp_563f2b19-0f47-40c0-8fff-c29b65f02314
6032024-09-06T23:35:30.075ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/ca24db04-f37b-40b1-83c2-23476bec5a4a/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
6042024-09-06T23:35:30.075ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/358fc814-3f2a-4b1a-aebb-894434d72d01/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
6052024-09-06T23:35:30.076ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 6228975 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:35852 req_id = 16ff8717-350b-4e88-b64e-491933f5c726 response_code = 204 uri = /omicron-zones
6062024-09-06T23:35:30.078ZINFOSledAgent (RSS): Waiting for rack time synchronization file = sled-agent/src/rack_setup/service.rs:652
6072024-09-06T23:35:30.108ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:632
6082024-09-06T23:35:30.108ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:41188
6092024-09-06T23:35:30.119ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_563f2b19-0f47-40c0-8fff-c29b65f02314': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3828
6102024-09-06T23:35:30.119ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:902 latency_us = 10413 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:41188 req_id = d02e82d1-4338-4619-beb3-725fae2e3a09 response_code = 500 uri = /timesync
6112024-09-06T23:35:30.120ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:682
6122024-09-06T23:35:30.231ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:632
6132024-09-06T23:35:30.231ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:54090
6142024-09-06T23:35:34.240ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_563f2b19-0f47-40c0-8fff-c29b65f02314': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3828
6152024-09-06T23:35:34.240ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:902 latency_us = 4009177 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:54090 req_id = f49b6aff-e481-4347-881e-c3c08ca0a283 response_code = 500 uri = /timesync
6162024-09-06T23:35:34.245ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:682
6172024-09-06T23:35:34.366ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:632
6182024-09-06T23:35:34.367ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:63471
6192024-09-06T23:35:34.375ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_563f2b19-0f47-40c0-8fff-c29b65f02314': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3828
6202024-09-06T23:35:34.375ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:902 latency_us = 8795 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:63471 req_id = e5a59232-8f17-41f4-9f01-594213f5c190 response_code = 500 uri = /timesync
6212024-09-06T23:35:34.378ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:682
6222024-09-06T23:35:34.493ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:632
6232024-09-06T23:35:34.493ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:37411
6242024-09-06T23:35:38.501ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_563f2b19-0f47-40c0-8fff-c29b65f02314': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3828
6252024-09-06T23:35:38.502ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:902 latency_us = 4009466 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:37411 req_id = 08b7b655-52aa-4ad5-a2b7-0982bd94db70 response_code = 500 uri = /timesync
6262024-09-06T23:35:38.504ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:682
6272024-09-06T23:35:38.658ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:632
6282024-09-06T23:35:38.658ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:45135
6292024-09-06T23:35:38.667ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8834 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:45135 req_id = 3aa7ec0b-3876-4849-957a-3003327e2311 response_code = 200 uri = /timesync
6302024-09-06T23:35:38.670ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1725665736.006434, correction: -0.009488566 } file = sled-agent/src/rack_setup/service.rs:660
6312024-09-06T23:35:38.670ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:682
6322024-09-06T23:35:39.000ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:632
6332024-09-06T23:35:39.000ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:62612
6342024-09-06T23:35:39.008ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8024 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:62612 req_id = b90c1f8f-190d-4728-95c9-23390ed5905b response_code = 200 uri = /timesync
6352024-09-06T23:35:39.010ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1725665736.006434, correction: -0.008561227 } file = sled-agent/src/rack_setup/service.rs:660
6362024-09-06T23:35:39.010ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:682
6372024-09-06T23:35:39.227ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:632
6382024-09-06T23:35:39.227ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:50478
6392024-09-06T23:35:39.236ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8608 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:50478 req_id = 7c34127e-3b45-4d12-bbea-da0c50bd1817 response_code = 200 uri = /timesync
6402024-09-06T23:35:39.239ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1725665736.006434, correction: -0.007944413 } file = sled-agent/src/rack_setup/service.rs:660
6412024-09-06T23:35:39.239ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:682
6422024-09-06T23:35:39.710ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:632
6432024-09-06T23:35:39.711ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:36868
6442024-09-06T23:35:39.720ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 9668 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:36868 req_id = 4a0a0ac2-24ae-4956-a32c-b6dd0627bda5 response_code = 200 uri = /timesync
6452024-09-06T23:35:39.723ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1725665736.006434, correction: -0.008316307 } file = sled-agent/src/rack_setup/service.rs:660
6462024-09-06T23:35:39.723ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:682
6472024-09-06T23:35:39.989ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:632
6482024-09-06T23:35:39.989ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:33856
6492024-09-06T23:35:39.998ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8978 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:33856 req_id = a3590a98-da3e-4ee2-8ca8-ed355cfa11f6 response_code = 200 uri = /timesync
6502024-09-06T23:35:40.001ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1725665736.006434, correction: -0.007561537 } file = sled-agent/src/rack_setup/service.rs:660
6512024-09-06T23:35:40.001ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:682
6522024-09-06T23:35:40.408ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:632
6532024-09-06T23:35:40.408ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:34716
6542024-09-06T23:35:40.427ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 19632 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:34716 req_id = ed5432d3-eb55-4f9d-97bf-c386ddbaca0c response_code = 200 uri = /timesync
6552024-09-06T23:35:40.430ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: true, ref_id: 395716850, ip_addr: 23.150.40.242, stratum: 3, ref_time: 1725665739.340688, correction: -0.006351076 } file = sled-agent/src/rack_setup/service.rs:660
6562024-09-06T23:35:40.430ZINFOSledAgent (RSS): Finished setting up Internal DNS and NTP file = sled-agent/src/rack_setup/service.rs:1299
6572024-09-06T23:35:40.463ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 26e8255e-2d98-41fa-b877-69c2893c27d0, underlay_address: fd00:1122:3344:1::1, filesystem_pool: Some(ZpoolName { id: 652da97e-d553-4457-bc42-d10c0c455967 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 652da97e-d553-4457-bc42-d10c0c455967 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: fcc84a23-e14a-4383-9764-158f0d5fc6c9, underlay_address: fd00:1122:3344:2::1, filesystem_pool: Some(ZpoolName { id: 1bae97fb-c4f0-4d17-b41c-409574fb9e6e (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 1bae97fb-c4f0-4d17-b41c-409574fb9e6e (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: a717f03d-27a8-4fde-9656-972b10665736, underlay_address: fd00:1122:3344:3::1, filesystem_pool: Some(ZpoolName { id: eae5bedf-a363-4cf3-b83a-efcb44445301 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: eae5bedf-a363-4cf3-b83a-efcb44445301 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 563f2b19-0f47-40c0-8fff-c29b65f02314, underlay_address: fd00:1122:3344:101::1b, filesystem_pool: Some(ZpoolName { id: eae5bedf-a363-4cf3-b83a-efcb44445301 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: acd43372-de8f-49b0-a1ec-6f7d2718bb22, kind: Service { id: 563f2b19-0f47-40c0-8fff-c29b65f02314 }, name: Name("ntp-563f2b19-0f47-40c0-8fff-c29b65f02314"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 202])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: f36b391d-1566-4909-bff8-8297bfb4c166, underlay_address: fd00:1122:3344:101::3, filesystem_pool: Some(ZpoolName { id: 652da97e-d553-4457-bc42-d10c0c455967 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 652da97e-d553-4457-bc42-d10c0c455967 (zpool), kind: External } } } }, OmicronZoneConfig { id: d878e8a1-acf2-4658-8ef3-2a29f89a5643, underlay_address: fd00:1122:3344:101::4, filesystem_pool: Some(ZpoolName { id: 1bae97fb-c4f0-4d17-b41c-409574fb9e6e (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 1bae97fb-c4f0-4d17-b41c-409574fb9e6e (zpool), kind: External } } } }, OmicronZoneConfig { id: 4fbe9fe2-68ea-45d4-ae9a-4765a236983f, underlay_address: fd00:1122:3344:101::5, filesystem_pool: Some(ZpoolName { id: eae5bedf-a363-4cf3-b83a-efcb44445301 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: eae5bedf-a363-4cf3-b83a-efcb44445301 (zpool), kind: External } } } }, OmicronZoneConfig { id: 50d89311-4ddf-44e9-94f4-8f8b68421caf, underlay_address: fd00:1122:3344:101::6, filesystem_pool: Some(ZpoolName { id: ab9a060c-a3d5-48de-ba57-beaa10d18520 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ab9a060c-a3d5-48de-ba57-beaa10d18520 (zpool), kind: External } } } }, OmicronZoneConfig { id: 4b4f9d05-21f6-4b48-9ca1-3f388ea217ce, underlay_address: fd00:1122:3344:101::7, filesystem_pool: Some(ZpoolName { id: fd8d5a1e-9723-41c1-aa2b-460a6959d6c7 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fd8d5a1e-9723-41c1-aa2b-460a6959d6c7 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:458 sled_address = [fd00:1122:3344:101::1]:12345
6582024-09-06T23:35:40.463ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:35339
6592024-09-06T23:35:40.463ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 26e8255e-2d98-41fa-b877-69c2893c27d0, dataset_name: DatasetName { pool_name: ZpoolName { id: 652da97e-d553-4457-bc42-d10c0c455967 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
6602024-09-06T23:35:40.463ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 26e8255e-2d98-41fa-b877-69c2893c27d0, dataset_name: DatasetName { pool_name: ZpoolName { id: 652da97e-d553-4457-bc42-d10c0c455967 (zpool), kind: External }, kind: InternalDns }, responder: ... }
6612024-09-06T23:35:40.480ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: fcc84a23-e14a-4383-9764-158f0d5fc6c9, dataset_name: DatasetName { pool_name: ZpoolName { id: 1bae97fb-c4f0-4d17-b41c-409574fb9e6e (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
6622024-09-06T23:35:40.480ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: fcc84a23-e14a-4383-9764-158f0d5fc6c9, dataset_name: DatasetName { pool_name: ZpoolName { id: 1bae97fb-c4f0-4d17-b41c-409574fb9e6e (zpool), kind: External }, kind: InternalDns }, responder: ... }
6632024-09-06T23:35:40.500ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: a717f03d-27a8-4fde-9656-972b10665736, dataset_name: DatasetName { pool_name: ZpoolName { id: eae5bedf-a363-4cf3-b83a-efcb44445301 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
6642024-09-06T23:35:40.500ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: a717f03d-27a8-4fde-9656-972b10665736, dataset_name: DatasetName { pool_name: ZpoolName { id: eae5bedf-a363-4cf3-b83a-efcb44445301 (zpool), kind: External }, kind: InternalDns }, responder: ... }
6652024-09-06T23:35:40.517ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f36b391d-1566-4909-bff8-8297bfb4c166, dataset_name: DatasetName { pool_name: ZpoolName { id: 652da97e-d553-4457-bc42-d10c0c455967 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
6662024-09-06T23:35:40.517ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: f36b391d-1566-4909-bff8-8297bfb4c166, dataset_name: DatasetName { pool_name: ZpoolName { id: 652da97e-d553-4457-bc42-d10c0c455967 (zpool), kind: External }, kind: Cockroach }, responder: ... }
6672024-09-06T23:35:40.569ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: d878e8a1-acf2-4658-8ef3-2a29f89a5643, dataset_name: DatasetName { pool_name: ZpoolName { id: 1bae97fb-c4f0-4d17-b41c-409574fb9e6e (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
6682024-09-06T23:35:40.569ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: d878e8a1-acf2-4658-8ef3-2a29f89a5643, dataset_name: DatasetName { pool_name: ZpoolName { id: 1bae97fb-c4f0-4d17-b41c-409574fb9e6e (zpool), kind: External }, kind: Cockroach }, responder: ... }
6692024-09-06T23:35:40.620ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 4fbe9fe2-68ea-45d4-ae9a-4765a236983f, dataset_name: DatasetName { pool_name: ZpoolName { id: eae5bedf-a363-4cf3-b83a-efcb44445301 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
6702024-09-06T23:35:40.620ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 4fbe9fe2-68ea-45d4-ae9a-4765a236983f, dataset_name: DatasetName { pool_name: ZpoolName { id: eae5bedf-a363-4cf3-b83a-efcb44445301 (zpool), kind: External }, kind: Cockroach }, responder: ... }
6712024-09-06T23:35:40.673ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 50d89311-4ddf-44e9-94f4-8f8b68421caf, dataset_name: DatasetName { pool_name: ZpoolName { id: ab9a060c-a3d5-48de-ba57-beaa10d18520 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
6722024-09-06T23:35:40.673ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 50d89311-4ddf-44e9-94f4-8f8b68421caf, dataset_name: DatasetName { pool_name: ZpoolName { id: ab9a060c-a3d5-48de-ba57-beaa10d18520 (zpool), kind: External }, kind: Cockroach }, responder: ... }
6732024-09-06T23:35:40.724ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 4b4f9d05-21f6-4b48-9ca1-3f388ea217ce, dataset_name: DatasetName { pool_name: ZpoolName { id: fd8d5a1e-9723-41c1-aa2b-460a6959d6c7 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
6742024-09-06T23:35:40.724ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 4b4f9d05-21f6-4b48-9ca1-3f388ea217ce, dataset_name: DatasetName { pool_name: ZpoolName { id: fd8d5a1e-9723-41c1-aa2b-460a6959d6c7 (zpool), kind: External }, kind: Cockroach }, responder: ... }
6752024-09-06T23:35:40.776ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
6762024-09-06T23:35:40.776ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/ca24db04-f37b-40b1-83c2-23476bec5a4a/config/omicron-zones.json file = common/src/ledger.rs:177
6772024-09-06T23:35:40.776ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/358fc814-3f2a-4b1a-aebb-894434d72d01/config/omicron-zones.json file = common/src/ledger.rs:177
6782024-09-06T23:35:40.776ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
6792024-09-06T23:35:40.858ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
6802024-09-06T23:35:40.858ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
6812024-09-06T23:35:40.858ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
6822024-09-06T23:35:40.858ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
6832024-09-06T23:35:40.858ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
6842024-09-06T23:35:40.947ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_4fbe9fe2-68ea-45d4-ae9a-4765a236983f file = illumos-utils/src/zone.rs:316
6852024-09-06T23:35:40.951ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_4b4f9d05-21f6-4b48-9ca1-3f388ea217ce file = illumos-utils/src/zone.rs:316
6862024-09-06T23:35:40.955ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_50d89311-4ddf-44e9-94f4-8f8b68421caf file = illumos-utils/src/zone.rs:316
6872024-09-06T23:35:40.959ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_d878e8a1-acf2-4658-8ef3-2a29f89a5643 file = illumos-utils/src/zone.rs:316
6882024-09-06T23:35:40.964ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_f36b391d-1566-4909-bff8-8297bfb4c166 file = illumos-utils/src/zone.rs:316
6892024-09-06T23:35:40.968ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_4fbe9fe2-68ea-45d4-ae9a-4765a236983f file = illumos-utils/src/zone.rs:354
6902024-09-06T23:35:40.972ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_4b4f9d05-21f6-4b48-9ca1-3f388ea217ce file = illumos-utils/src/zone.rs:354
6912024-09-06T23:35:40.977ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_50d89311-4ddf-44e9-94f4-8f8b68421caf file = illumos-utils/src/zone.rs:354
6922024-09-06T23:35:40.982ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_d878e8a1-acf2-4658-8ef3-2a29f89a5643 file = illumos-utils/src/zone.rs:354
6932024-09-06T23:35:40.987ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_f36b391d-1566-4909-bff8-8297bfb4c166 file = illumos-utils/src/zone.rs:354
6942024-09-06T23:35:43.948ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_d878e8a1-acf2-4658-8ef3-2a29f89a5643: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService8'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::4'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::4]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='d878e8a1-acf2-4658-8ef3-2a29f89a5643'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::4]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::4]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
6952024-09-06T23:35:43.949ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_d878e8a1-acf2-4658-8ef3-2a29f89a5643 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_d878e8a1-acf2-4658-8ef3-2a29f89a5643
6962024-09-06T23:35:43.954ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_f36b391d-1566-4909-bff8-8297bfb4c166: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService9'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::3'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::3]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='f36b391d-1566-4909-bff8-8297bfb4c166'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::3]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::3]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
6972024-09-06T23:35:43.954ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_f36b391d-1566-4909-bff8-8297bfb4c166 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_f36b391d-1566-4909-bff8-8297bfb4c166
6982024-09-06T23:35:43.973ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_4b4f9d05-21f6-4b48-9ca1-3f388ea217ce: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService6'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::7'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::7]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='4b4f9d05-21f6-4b48-9ca1-3f388ea217ce'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::7]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::7]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
6992024-09-06T23:35:43.973ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_4b4f9d05-21f6-4b48-9ca1-3f388ea217ce zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_4b4f9d05-21f6-4b48-9ca1-3f388ea217ce
7002024-09-06T23:35:43.979ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_4fbe9fe2-68ea-45d4-ae9a-4765a236983f: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService5'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::5]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='4fbe9fe2-68ea-45d4-ae9a-4765a236983f'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::5]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::5]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
7012024-09-06T23:35:43.979ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_4fbe9fe2-68ea-45d4-ae9a-4765a236983f zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_4fbe9fe2-68ea-45d4-ae9a-4765a236983f
7022024-09-06T23:35:43.987ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_50d89311-4ddf-44e9-94f4-8f8b68421caf: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService7'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::6]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='50d89311-4ddf-44e9-94f4-8f8b68421caf'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::6]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::6]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
7032024-09-06T23:35:43.987ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_50d89311-4ddf-44e9-94f4-8f8b68421caf zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_50d89311-4ddf-44e9-94f4-8f8b68421caf
7042024-09-06T23:35:44.091ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d878e8a1-acf2-4658-8ef3-2a29f89a5643") failed: Property not found. retry in 73.790647ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d878e8a1-acf2-4658-8ef3-2a29f89a5643
7052024-09-06T23:35:44.108ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f36b391d-1566-4909-bff8-8297bfb4c166") failed: Property not found. retry in 45.125019ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f36b391d-1566-4909-bff8-8297bfb4c166
7062024-09-06T23:35:44.116ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4fbe9fe2-68ea-45d4-ae9a-4765a236983f") failed: Property not found. retry in 38.49497ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4fbe9fe2-68ea-45d4-ae9a-4765a236983f
7072024-09-06T23:35:44.122ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4b4f9d05-21f6-4b48-9ca1-3f388ea217ce") failed: Property not found. retry in 47.847911ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4b4f9d05-21f6-4b48-9ca1-3f388ea217ce
7082024-09-06T23:35:44.128ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_50d89311-4ddf-44e9-94f4-8f8b68421caf") failed: Property not found. retry in 55.355659ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_50d89311-4ddf-44e9-94f4-8f8b68421caf
7092024-09-06T23:35:44.172ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4fbe9fe2-68ea-45d4-ae9a-4765a236983f") failed: Property not found. retry in 102.843332ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4fbe9fe2-68ea-45d4-ae9a-4765a236983f
7102024-09-06T23:35:44.322ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f36b391d-1566-4909-bff8-8297bfb4c166") failed: Property not found. retry in 67.592893ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f36b391d-1566-4909-bff8-8297bfb4c166
7112024-09-06T23:35:44.365ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4fbe9fe2-68ea-45d4-ae9a-4765a236983f") failed: Property not found. retry in 205.112926ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4fbe9fe2-68ea-45d4-ae9a-4765a236983f
7122024-09-06T23:35:44.414ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4b4f9d05-21f6-4b48-9ca1-3f388ea217ce") failed: Property not found. retry in 122.366135ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4b4f9d05-21f6-4b48-9ca1-3f388ea217ce
7132024-09-06T23:35:44.965ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_50d89311-4ddf-44e9-94f4-8f8b68421caf") failed: Property not found. retry in 91.977573ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_50d89311-4ddf-44e9-94f4-8f8b68421caf
7142024-09-06T23:35:45.134ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d878e8a1-acf2-4658-8ef3-2a29f89a5643") failed: Property not found. retry in 61.630248ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d878e8a1-acf2-4658-8ef3-2a29f89a5643
7152024-09-06T23:35:45.173ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f36b391d-1566-4909-bff8-8297bfb4c166") failed: Property not found. retry in 130.098218ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f36b391d-1566-4909-bff8-8297bfb4c166
7162024-09-06T23:35:45.592ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4fbe9fe2-68ea-45d4-ae9a-4765a236983f") failed: Property not found. retry in 306.001092ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4fbe9fe2-68ea-45d4-ae9a-4765a236983f
7172024-09-06T23:35:45.670ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4b4f9d05-21f6-4b48-9ca1-3f388ea217ce") failed: Property not found. retry in 176.04215ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4b4f9d05-21f6-4b48-9ca1-3f388ea217ce
7182024-09-06T23:35:45.888ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_50d89311-4ddf-44e9-94f4-8f8b68421caf") failed: Property not found. retry in 118.274812ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_50d89311-4ddf-44e9-94f4-8f8b68421caf
7192024-09-06T23:35:46.880ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d878e8a1-acf2-4658-8ef3-2a29f89a5643") failed: Property not found. retry in 286.226894ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d878e8a1-acf2-4658-8ef3-2a29f89a5643
7202024-09-06T23:35:46.929ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f36b391d-1566-4909-bff8-8297bfb4c166") failed: Property not found. retry in 310.460167ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f36b391d-1566-4909-bff8-8297bfb4c166
7212024-09-06T23:35:46.940ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4fbe9fe2-68ea-45d4-ae9a-4765a236983f") failed: Property not found. retry in 1.116832116s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4fbe9fe2-68ea-45d4-ae9a-4765a236983f
7222024-09-06T23:35:46.949ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4b4f9d05-21f6-4b48-9ca1-3f388ea217ce") failed: Property not found. retry in 339.935993ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4b4f9d05-21f6-4b48-9ca1-3f388ea217ce
7232024-09-06T23:35:46.959ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_50d89311-4ddf-44e9-94f4-8f8b68421caf") failed: Property not found. retry in 361.663774ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_50d89311-4ddf-44e9-94f4-8f8b68421caf
7242024-09-06T23:35:47.177ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d878e8a1-acf2-4658-8ef3-2a29f89a5643") failed: Property not found. retry in 423.187331ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d878e8a1-acf2-4658-8ef3-2a29f89a5643
7252024-09-06T23:35:47.249ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f36b391d-1566-4909-bff8-8297bfb4c166") failed: Property not found. retry in 1.171611339s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f36b391d-1566-4909-bff8-8297bfb4c166
7262024-09-06T23:35:47.300ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4b4f9d05-21f6-4b48-9ca1-3f388ea217ce") failed: Property not found. retry in 1.068513598s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4b4f9d05-21f6-4b48-9ca1-3f388ea217ce
7272024-09-06T23:35:47.333ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_50d89311-4ddf-44e9-94f4-8f8b68421caf") failed: Property not found. retry in 804.940469ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_50d89311-4ddf-44e9-94f4-8f8b68421caf
7282024-09-06T23:35:47.613ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d878e8a1-acf2-4658-8ef3-2a29f89a5643") failed: Property not found. retry in 598.310567ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d878e8a1-acf2-4658-8ef3-2a29f89a5643
7292024-09-06T23:35:48.069ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4fbe9fe2-68ea-45d4-ae9a-4765a236983f") failed: Property not found. retry in 1.087552605s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4fbe9fe2-68ea-45d4-ae9a-4765a236983f
7302024-09-06T23:35:48.148ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_50d89311-4ddf-44e9-94f4-8f8b68421caf") failed: Property not found. retry in 1.365580987s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_50d89311-4ddf-44e9-94f4-8f8b68421caf
7312024-09-06T23:35:48.221ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d878e8a1-acf2-4658-8ef3-2a29f89a5643") failed: Property not found. retry in 1.325671621s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d878e8a1-acf2-4658-8ef3-2a29f89a5643
7322024-09-06T23:35:48.380ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4b4f9d05-21f6-4b48-9ca1-3f388ea217ce") failed: Property not found. retry in 1.481707305s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4b4f9d05-21f6-4b48-9ca1-3f388ea217ce
7332024-09-06T23:35:48.432ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f36b391d-1566-4909-bff8-8297bfb4c166") failed: Property not found. retry in 580.608491ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f36b391d-1566-4909-bff8-8297bfb4c166
7342024-09-06T23:35:49.023ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f36b391d-1566-4909-bff8-8297bfb4c166") failed: Property not found. retry in 614.559503ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f36b391d-1566-4909-bff8-8297bfb4c166
7352024-09-06T23:35:49.167ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4fbe9fe2-68ea-45d4-ae9a-4765a236983f") failed: Property not found. retry in 1.453912731s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4fbe9fe2-68ea-45d4-ae9a-4765a236983f
7362024-09-06T23:35:49.268ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 320.189402ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c
7372024-09-06T23:35:49.525ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_50d89311-4ddf-44e9-94f4-8f8b68421caf") failed: Property not found. retry in 1.277344071s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_50d89311-4ddf-44e9-94f4-8f8b68421caf
7382024-09-06T23:35:49.557ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d878e8a1-acf2-4658-8ef3-2a29f89a5643") failed: Property not found. retry in 633.016283ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d878e8a1-acf2-4658-8ef3-2a29f89a5643
7392024-09-06T23:35:49.648ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f36b391d-1566-4909-bff8-8297bfb4c166") failed: Property not found. retry in 976.092876ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f36b391d-1566-4909-bff8-8297bfb4c166
7402024-09-06T23:35:49.871ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4b4f9d05-21f6-4b48-9ca1-3f388ea217ce") failed: Property not found. retry in 970.0554ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4b4f9d05-21f6-4b48-9ca1-3f388ea217ce
7412024-09-06T23:35:50.199ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d878e8a1-acf2-4658-8ef3-2a29f89a5643") failed: Property not found. retry in 1.107822907s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d878e8a1-acf2-4658-8ef3-2a29f89a5643
7422024-09-06T23:35:50.515ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:61402
7432024-09-06T23:35:50.516ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 704 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:61402 req_id = d7e7bea5-1fc4-4f0c-a233-9764f2bf6d3d response_code = 200 uri = /baseboard
7442024-09-06T23:35:50.631ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4fbe9fe2-68ea-45d4-ae9a-4765a236983f") failed: Property not found. retry in 518.502946ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4fbe9fe2-68ea-45d4-ae9a-4765a236983f
7452024-09-06T23:35:50.641ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f36b391d-1566-4909-bff8-8297bfb4c166") failed: Property not found. retry in 1.044265287s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f36b391d-1566-4909-bff8-8297bfb4c166
7462024-09-06T23:35:50.812ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_50d89311-4ddf-44e9-94f4-8f8b68421caf") failed: Property not found. retry in 1.22805009s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_50d89311-4ddf-44e9-94f4-8f8b68421caf
7472024-09-06T23:35:50.851ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4b4f9d05-21f6-4b48-9ca1-3f388ea217ce") failed: Property not found. retry in 1.368732628s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4b4f9d05-21f6-4b48-9ca1-3f388ea217ce
7482024-09-06T23:35:51.160ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4fbe9fe2-68ea-45d4-ae9a-4765a236983f") failed: Property not found. retry in 737.347472ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4fbe9fe2-68ea-45d4-ae9a-4765a236983f
7492024-09-06T23:35:51.316ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d878e8a1-acf2-4658-8ef3-2a29f89a5643") failed: Property not found. retry in 1.412351884s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d878e8a1-acf2-4658-8ef3-2a29f89a5643
7502024-09-06T23:35:51.695ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f36b391d-1566-4909-bff8-8297bfb4c166") failed: Property not found. retry in 965.969012ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f36b391d-1566-4909-bff8-8297bfb4c166
7512024-09-06T23:35:51.908ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4fbe9fe2-68ea-45d4-ae9a-4765a236983f") failed: Property not found. retry in 921.090846ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4fbe9fe2-68ea-45d4-ae9a-4765a236983f
7522024-09-06T23:35:52.051ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_50d89311-4ddf-44e9-94f4-8f8b68421caf") failed: Property not found. retry in 1.4631219s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_50d89311-4ddf-44e9-94f4-8f8b68421caf
7532024-09-06T23:35:52.230ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4b4f9d05-21f6-4b48-9ca1-3f388ea217ce") failed: Property not found. retry in 775.462938ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4b4f9d05-21f6-4b48-9ca1-3f388ea217ce
7542024-09-06T23:35:52.674ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f36b391d-1566-4909-bff8-8297bfb4c166") failed: Property not found. retry in 674.781922ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f36b391d-1566-4909-bff8-8297bfb4c166
7552024-09-06T23:35:52.737ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d878e8a1-acf2-4658-8ef3-2a29f89a5643") failed: Property not found. retry in 518.01829ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d878e8a1-acf2-4658-8ef3-2a29f89a5643
7562024-09-06T23:35:52.839ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4fbe9fe2-68ea-45d4-ae9a-4765a236983f") failed: Property not found. retry in 1.418513606s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4fbe9fe2-68ea-45d4-ae9a-4765a236983f
7572024-09-06T23:35:53.019ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4b4f9d05-21f6-4b48-9ca1-3f388ea217ce") failed: Property not found. retry in 650.209393ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4b4f9d05-21f6-4b48-9ca1-3f388ea217ce
7582024-09-06T23:35:53.265ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d878e8a1-acf2-4658-8ef3-2a29f89a5643") failed: Property not found. retry in 746.466448ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d878e8a1-acf2-4658-8ef3-2a29f89a5643
7592024-09-06T23:35:53.357ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f36b391d-1566-4909-bff8-8297bfb4c166") failed: Property not found. retry in 1.339086681s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f36b391d-1566-4909-bff8-8297bfb4c166
7602024-09-06T23:35:53.522ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_50d89311-4ddf-44e9-94f4-8f8b68421caf") failed: Property not found. retry in 755.553857ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_50d89311-4ddf-44e9-94f4-8f8b68421caf
7612024-09-06T23:35:53.678ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4b4f9d05-21f6-4b48-9ca1-3f388ea217ce") failed: Property not found. retry in 957.032409ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4b4f9d05-21f6-4b48-9ca1-3f388ea217ce
7622024-09-06T23:35:54.019ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d878e8a1-acf2-4658-8ef3-2a29f89a5643") failed: Property not found. retry in 1.392860046s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d878e8a1-acf2-4658-8ef3-2a29f89a5643
7632024-09-06T23:35:54.266ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4fbe9fe2-68ea-45d4-ae9a-4765a236983f") failed: Property not found. retry in 1.364591041s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4fbe9fe2-68ea-45d4-ae9a-4765a236983f
7642024-09-06T23:35:54.286ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_50d89311-4ddf-44e9-94f4-8f8b68421caf") failed: Property not found. retry in 1.378854967s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_50d89311-4ddf-44e9-94f4-8f8b68421caf
7652024-09-06T23:35:54.645ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4b4f9d05-21f6-4b48-9ca1-3f388ea217ce") failed: Property not found. retry in 578.256621ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4b4f9d05-21f6-4b48-9ca1-3f388ea217ce
7662024-09-06T23:35:54.705ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f36b391d-1566-4909-bff8-8297bfb4c166") failed: Property not found. retry in 832.726829ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f36b391d-1566-4909-bff8-8297bfb4c166
7672024-09-06T23:35:55.233ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4b4f9d05-21f6-4b48-9ca1-3f388ea217ce") failed: Property not found. retry in 774.364155ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4b4f9d05-21f6-4b48-9ca1-3f388ea217ce
7682024-09-06T23:35:55.421ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d878e8a1-acf2-4658-8ef3-2a29f89a5643") failed: Property not found. retry in 1.164281418s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d878e8a1-acf2-4658-8ef3-2a29f89a5643
7692024-09-06T23:35:55.546ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f36b391d-1566-4909-bff8-8297bfb4c166") failed: Property not found. retry in 588.520306ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f36b391d-1566-4909-bff8-8297bfb4c166
7702024-09-06T23:35:55.639ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4fbe9fe2-68ea-45d4-ae9a-4765a236983f") failed: Property not found. retry in 523.667296ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4fbe9fe2-68ea-45d4-ae9a-4765a236983f
7712024-09-06T23:35:55.673ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_50d89311-4ddf-44e9-94f4-8f8b68421caf") failed: Property not found. retry in 805.675925ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_50d89311-4ddf-44e9-94f4-8f8b68421caf
7722024-09-06T23:35:56.019ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4b4f9d05-21f6-4b48-9ca1-3f388ea217ce") failed: Property not found. retry in 1.376600598s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4b4f9d05-21f6-4b48-9ca1-3f388ea217ce
7732024-09-06T23:35:57.419ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3226 zone = oxz_cockroachdb_4fbe9fe2-68ea-45d4-ae9a-4765a236983f
7742024-09-06T23:35:57.419ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3226 zone = oxz_cockroachdb_4b4f9d05-21f6-4b48-9ca1-3f388ea217ce
7752024-09-06T23:35:57.419ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3226 zone = oxz_cockroachdb_50d89311-4ddf-44e9-94f4-8f8b68421caf
7762024-09-06T23:35:57.419ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3226 zone = oxz_cockroachdb_d878e8a1-acf2-4658-8ef3-2a29f89a5643
7772024-09-06T23:35:57.419ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3226 zone = oxz_cockroachdb_f36b391d-1566-4909-bff8-8297bfb4c166
7782024-09-06T23:35:57.419ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/ca24db04-f37b-40b1-83c2-23476bec5a4a/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
7792024-09-06T23:35:57.419ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/358fc814-3f2a-4b1a-aebb-894434d72d01/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
7802024-09-06T23:35:57.420ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 16962719 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:35339 req_id = aac05567-4f9d-4102-af5e-a974a86f40d1 response_code = 204 uri = /omicron-zones
7812024-09-06T23:35:57.454ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:45796
7822024-09-06T23:35:57.491ZINFOSledAgent (ServiceManager): Initializing CRDB Cluster - sending request to [fd00:1122:3344:101::7]:32221 file = sled-agent/src/services.rs:3662
7832024-09-06T23:36:18.077ZINFOSledAgent (ServiceManager): Formatting CRDB file = sled-agent/src/services.rs:3680
7842024-09-06T23:36:25.036ZINFOSledAgent (ServiceManager): Formatting CRDB - Completed file = sled-agent/src/services.rs:3703
7852024-09-06T23:36:25.036ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 27581284 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:45796 req_id = fccaf518-d477-4998-b2ee-dd0b0ec6e044 response_code = 204 uri = /cockroachdb
7862024-09-06T23:36:25.037ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:47365
7872024-09-06T23:36:25.037ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 168 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:47365 req_id = cf7c26ef-b4c5-4e72-8c88-4e545704cb8f response_code = 200 uri = /baseboard
7882024-09-06T23:36:25.041ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 693.537337ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c
7892024-09-06T23:36:25.073ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(5), zones: [OmicronZoneConfig { id: 26e8255e-2d98-41fa-b877-69c2893c27d0, underlay_address: fd00:1122:3344:1::1, filesystem_pool: Some(ZpoolName { id: 652da97e-d553-4457-bc42-d10c0c455967 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 652da97e-d553-4457-bc42-d10c0c455967 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: fcc84a23-e14a-4383-9764-158f0d5fc6c9, underlay_address: fd00:1122:3344:2::1, filesystem_pool: Some(ZpoolName { id: 1bae97fb-c4f0-4d17-b41c-409574fb9e6e (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 1bae97fb-c4f0-4d17-b41c-409574fb9e6e (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: a717f03d-27a8-4fde-9656-972b10665736, underlay_address: fd00:1122:3344:3::1, filesystem_pool: Some(ZpoolName { id: eae5bedf-a363-4cf3-b83a-efcb44445301 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: eae5bedf-a363-4cf3-b83a-efcb44445301 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 563f2b19-0f47-40c0-8fff-c29b65f02314, underlay_address: fd00:1122:3344:101::1b, filesystem_pool: Some(ZpoolName { id: eae5bedf-a363-4cf3-b83a-efcb44445301 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: acd43372-de8f-49b0-a1ec-6f7d2718bb22, kind: Service { id: 563f2b19-0f47-40c0-8fff-c29b65f02314 }, name: Name("ntp-563f2b19-0f47-40c0-8fff-c29b65f02314"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 202])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: f36b391d-1566-4909-bff8-8297bfb4c166, underlay_address: fd00:1122:3344:101::3, filesystem_pool: Some(ZpoolName { id: 652da97e-d553-4457-bc42-d10c0c455967 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 652da97e-d553-4457-bc42-d10c0c455967 (zpool), kind: External } } } }, OmicronZoneConfig { id: d878e8a1-acf2-4658-8ef3-2a29f89a5643, underlay_address: fd00:1122:3344:101::4, filesystem_pool: Some(ZpoolName { id: 1bae97fb-c4f0-4d17-b41c-409574fb9e6e (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 1bae97fb-c4f0-4d17-b41c-409574fb9e6e (zpool), kind: External } } } }, OmicronZoneConfig { id: 4fbe9fe2-68ea-45d4-ae9a-4765a236983f, underlay_address: fd00:1122:3344:101::5, filesystem_pool: Some(ZpoolName { id: eae5bedf-a363-4cf3-b83a-efcb44445301 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: eae5bedf-a363-4cf3-b83a-efcb44445301 (zpool), kind: External } } } }, OmicronZoneConfig { id: 50d89311-4ddf-44e9-94f4-8f8b68421caf, underlay_address: fd00:1122:3344:101::6, filesystem_pool: Some(ZpoolName { id: ab9a060c-a3d5-48de-ba57-beaa10d18520 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ab9a060c-a3d5-48de-ba57-beaa10d18520 (zpool), kind: External } } } }, OmicronZoneConfig { id: 4b4f9d05-21f6-4b48-9ca1-3f388ea217ce, underlay_address: fd00:1122:3344:101::7, filesystem_pool: Some(ZpoolName { id: fd8d5a1e-9723-41c1-aa2b-460a6959d6c7 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fd8d5a1e-9723-41c1-aa2b-460a6959d6c7 (zpool), kind: External } } } }, OmicronZoneConfig { id: 5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6, underlay_address: fd00:1122:3344:101::8, filesystem_pool: Some(ZpoolName { id: 652da97e-d553-4457-bc42-d10c0c455967 (zpool), kind: External }), zone_type: ExternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 652da97e-d553-4457-bc42-d10c0c455967 (zpool), kind: External } }, http_address: [fd00:1122:3344:101::8]:5353, dns_address: 10.151.2.164:53, nic: NetworkInterface { id: 183029ac-b909-4e34-9803-fdf7533f1a8f, kind: Service { id: 5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6 }, name: Name("external-dns-5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 179, 238])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } } }, OmicronZoneConfig { id: 195e4e82-1cc4-4f3b-a893-c1474bd26575, underlay_address: fd00:1122:3344:101::9, filesystem_pool: Some(ZpoolName { id: 1bae97fb-c4f0-4d17-b41c-409574fb9e6e (zpool), kind: External }), zone_type: ExternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 1bae97fb-c4f0-4d17-b41c-409574fb9e6e (zpool), kind: External } }, http_address: [fd00:1122:3344:101::9]:5353, dns_address: 10.151.2.165:53, nic: NetworkInterface { id: 7ef66645-e8c6-4c80-8f48-af73dfa3e97b, kind: Service { id: 195e4e82-1cc4-4f3b-a893-c1474bd26575 }, name: Name("external-dns-195e4e82-1cc4-4f3b-a893-c1474bd26575"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 212, 58])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } } }, OmicronZoneConfig { id: 9ad0f243-aeb1-455d-97ad-7dc9ca9c417c, underlay_address: fd00:1122:3344:101::a, filesystem_pool: Some(ZpoolName { id: cc3f0ccd-963c-4d0c-843d-e8810b230566 (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::a]:12221, external_ip: 10.151.2.166, nic: NetworkInterface { id: 41a43463-6084-4d04-ad1b-f1d1ddccceaf, kind: Service { id: 9ad0f243-aeb1-455d-97ad-7dc9ca9c417c }, name: Name("nexus-9ad0f243-aeb1-455d-97ad-7dc9ca9c417c"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 248, 203])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: 6c363cb4-229c-47a8-9d81-abd1714293f2, underlay_address: fd00:1122:3344:101::b, filesystem_pool: Some(ZpoolName { id: f8e03f3f-0403-4c51-926a-d5ce11d8408e (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::b]:12221, external_ip: 10.151.2.167, nic: NetworkInterface { id: ce58de05-7c50-42fb-bdcb-23c435be246c, kind: Service { id: 6c363cb4-229c-47a8-9d81-abd1714293f2 }, name: Name("nexus-6c363cb4-229c-47a8-9d81-abd1714293f2"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 218, 98])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: cb51aad1-b6ab-44e1-8492-82c028d74c7f, underlay_address: fd00:1122:3344:101::c, filesystem_pool: Some(ZpoolName { id: 907237be-faec-4233-8c3e-1918ec810764 (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::c]:12221, external_ip: 10.151.2.168, nic: NetworkInterface { id: a0534026-8f14-4794-ba39-35fb89d6d21c, kind: Service { id: cb51aad1-b6ab-44e1-8492-82c028d74c7f }, name: Name("nexus-cb51aad1-b6ab-44e1-8492-82c028d74c7f"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 170, 148])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: bc144d23-6ff5-421e-a179-41beb5b46c0f, underlay_address: fd00:1122:3344:101::d, filesystem_pool: Some(ZpoolName { id: f8e03f3f-0403-4c51-926a-d5ce11d8408e (zpool), kind: External }), zone_type: Oximeter { address: [fd00:1122:3344:101::d]:12223 } }, OmicronZoneConfig { id: 2ac4364c-7933-4568-bbab-cf3f9ec0ff29, underlay_address: fd00:1122:3344:101::e, filesystem_pool: Some(ZpoolName { id: 652da97e-d553-4457-bc42-d10c0c455967 (zpool), kind: External }), zone_type: Clickhouse { address: [fd00:1122:3344:101::e]:8123, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 652da97e-d553-4457-bc42-d10c0c455967 (zpool), kind: External } } } }, OmicronZoneConfig { id: 072c2fe9-5a06-4344-a93a-991a46098bd3, underlay_address: fd00:1122:3344:101::f, filesystem_pool: Some(ZpoolName { id: fd8d5a1e-9723-41c1-aa2b-460a6959d6c7 (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::f]:17000 } }, OmicronZoneConfig { id: b047cd23-d622-477e-885d-d1b3824512de, underlay_address: fd00:1122:3344:101::10, filesystem_pool: Some(ZpoolName { id: 907237be-faec-4233-8c3e-1918ec810764 (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::10]:17000 } }, OmicronZoneConfig { id: 764a9699-9635-421f-88c6-7f14971487fe, underlay_address: fd00:1122:3344:101::11, filesystem_pool: Some(ZpoolName { id: 907237be-faec-4233-8c3e-1918ec810764 (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::11]:17000 } }, OmicronZoneConfig { id: e273cca1-6e0d-4509-842e-40a51b28e236, underlay_address: fd00:1122:3344:101::12, filesystem_pool: Some(ZpoolName { id: 652da97e-d553-4457-bc42-d10c0c455967 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::12]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 652da97e-d553-4457-bc42-d10c0c455967 (zpool), kind: External } } } }, OmicronZoneConfig { id: 2adfe4a1-4baf-4cb8-a3d2-e6907b84545a, underlay_address: fd00:1122:3344:101::13, filesystem_pool: Some(ZpoolName { id: 1bae97fb-c4f0-4d17-b41c-409574fb9e6e (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::13]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 1bae97fb-c4f0-4d17-b41c-409574fb9e6e (zpool), kind: External } } } }, OmicronZoneConfig { id: dbf82130-678a-44d4-9dd9-8a8be286ccf6, underlay_address: fd00:1122:3344:101::14, filesystem_pool: Some(ZpoolName { id: eae5bedf-a363-4cf3-b83a-efcb44445301 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::14]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: eae5bedf-a363-4cf3-b83a-efcb44445301 (zpool), kind: External } } } }, OmicronZoneConfig { id: 0a5539c1-7e26-4e50-b26a-60a137388ce1, underlay_address: fd00:1122:3344:101::15, filesystem_pool: Some(ZpoolName { id: ab9a060c-a3d5-48de-ba57-beaa10d18520 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::15]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ab9a060c-a3d5-48de-ba57-beaa10d18520 (zpool), kind: External } } } }, OmicronZoneConfig { id: cea6def4-eb13-4381-bb48-7596d86978d4, underlay_address: fd00:1122:3344:101::16, filesystem_pool: Some(ZpoolName { id: fd8d5a1e-9723-41c1-aa2b-460a6959d6c7 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::16]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fd8d5a1e-9723-41c1-aa2b-460a6959d6c7 (zpool), kind: External } } } }, OmicronZoneConfig { id: ebce0088-396f-4300-91e9-19979170faa6, underlay_address: fd00:1122:3344:101::17, filesystem_pool: Some(ZpoolName { id: b79af253-dd33-4fe7-98c1-790c50e27394 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::17]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: b79af253-dd33-4fe7-98c1-790c50e27394 (zpool), kind: External } } } }, OmicronZoneConfig { id: fa9373bc-3206-40d0-b421-9f54e256513c, underlay_address: fd00:1122:3344:101::18, filesystem_pool: Some(ZpoolName { id: cc3f0ccd-963c-4d0c-843d-e8810b230566 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::18]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: cc3f0ccd-963c-4d0c-843d-e8810b230566 (zpool), kind: External } } } }, OmicronZoneConfig { id: 6bc5dbb5-a884-4ae2-af09-70bc587bd76b, underlay_address: fd00:1122:3344:101::19, filesystem_pool: Some(ZpoolName { id: f8e03f3f-0403-4c51-926a-d5ce11d8408e (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::19]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: f8e03f3f-0403-4c51-926a-d5ce11d8408e (zpool), kind: External } } } }, OmicronZoneConfig { id: c1d2ae9b-e921-4256-b95c-a29d6126fd66, underlay_address: fd00:1122:3344:101::1a, filesystem_pool: Some(ZpoolName { id: 907237be-faec-4233-8c3e-1918ec810764 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::1a]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 907237be-faec-4233-8c3e-1918ec810764 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:458 sled_address = [fd00:1122:3344:101::1]:12345
7902024-09-06T23:36:25.073ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:61861
7912024-09-06T23:36:25.073ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 26e8255e-2d98-41fa-b877-69c2893c27d0, dataset_name: DatasetName { pool_name: ZpoolName { id: 652da97e-d553-4457-bc42-d10c0c455967 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
7922024-09-06T23:36:25.073ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 26e8255e-2d98-41fa-b877-69c2893c27d0, dataset_name: DatasetName { pool_name: ZpoolName { id: 652da97e-d553-4457-bc42-d10c0c455967 (zpool), kind: External }, kind: InternalDns }, responder: ... }
7932024-09-06T23:36:25.096ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: fcc84a23-e14a-4383-9764-158f0d5fc6c9, dataset_name: DatasetName { pool_name: ZpoolName { id: 1bae97fb-c4f0-4d17-b41c-409574fb9e6e (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
7942024-09-06T23:36:25.096ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: fcc84a23-e14a-4383-9764-158f0d5fc6c9, dataset_name: DatasetName { pool_name: ZpoolName { id: 1bae97fb-c4f0-4d17-b41c-409574fb9e6e (zpool), kind: External }, kind: InternalDns }, responder: ... }
7952024-09-06T23:36:25.117ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: a717f03d-27a8-4fde-9656-972b10665736, dataset_name: DatasetName { pool_name: ZpoolName { id: eae5bedf-a363-4cf3-b83a-efcb44445301 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
7962024-09-06T23:36:25.117ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: a717f03d-27a8-4fde-9656-972b10665736, dataset_name: DatasetName { pool_name: ZpoolName { id: eae5bedf-a363-4cf3-b83a-efcb44445301 (zpool), kind: External }, kind: InternalDns }, responder: ... }
7972024-09-06T23:36:25.136ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f36b391d-1566-4909-bff8-8297bfb4c166, dataset_name: DatasetName { pool_name: ZpoolName { id: 652da97e-d553-4457-bc42-d10c0c455967 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
7982024-09-06T23:36:25.136ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: f36b391d-1566-4909-bff8-8297bfb4c166, dataset_name: DatasetName { pool_name: ZpoolName { id: 652da97e-d553-4457-bc42-d10c0c455967 (zpool), kind: External }, kind: Cockroach }, responder: ... }
7992024-09-06T23:36:25.156ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: d878e8a1-acf2-4658-8ef3-2a29f89a5643, dataset_name: DatasetName { pool_name: ZpoolName { id: 1bae97fb-c4f0-4d17-b41c-409574fb9e6e (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
8002024-09-06T23:36:25.156ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: d878e8a1-acf2-4658-8ef3-2a29f89a5643, dataset_name: DatasetName { pool_name: ZpoolName { id: 1bae97fb-c4f0-4d17-b41c-409574fb9e6e (zpool), kind: External }, kind: Cockroach }, responder: ... }
8012024-09-06T23:36:25.177ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 4fbe9fe2-68ea-45d4-ae9a-4765a236983f, dataset_name: DatasetName { pool_name: ZpoolName { id: eae5bedf-a363-4cf3-b83a-efcb44445301 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
8022024-09-06T23:36:25.177ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 4fbe9fe2-68ea-45d4-ae9a-4765a236983f, dataset_name: DatasetName { pool_name: ZpoolName { id: eae5bedf-a363-4cf3-b83a-efcb44445301 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8032024-09-06T23:36:25.196ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 50d89311-4ddf-44e9-94f4-8f8b68421caf, dataset_name: DatasetName { pool_name: ZpoolName { id: ab9a060c-a3d5-48de-ba57-beaa10d18520 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
8042024-09-06T23:36:25.196ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 50d89311-4ddf-44e9-94f4-8f8b68421caf, dataset_name: DatasetName { pool_name: ZpoolName { id: ab9a060c-a3d5-48de-ba57-beaa10d18520 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8052024-09-06T23:36:25.216ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 4b4f9d05-21f6-4b48-9ca1-3f388ea217ce, dataset_name: DatasetName { pool_name: ZpoolName { id: fd8d5a1e-9723-41c1-aa2b-460a6959d6c7 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
8062024-09-06T23:36:25.216ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 4b4f9d05-21f6-4b48-9ca1-3f388ea217ce, dataset_name: DatasetName { pool_name: ZpoolName { id: fd8d5a1e-9723-41c1-aa2b-460a6959d6c7 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8072024-09-06T23:36:25.234ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6, dataset_name: DatasetName { pool_name: ZpoolName { id: 652da97e-d553-4457-bc42-d10c0c455967 (zpool), kind: External }, kind: ExternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
8082024-09-06T23:36:25.234ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6, dataset_name: DatasetName { pool_name: ZpoolName { id: 652da97e-d553-4457-bc42-d10c0c455967 (zpool), kind: External }, kind: ExternalDns }, responder: ... }
8092024-09-06T23:36:25.355ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 195e4e82-1cc4-4f3b-a893-c1474bd26575, dataset_name: DatasetName { pool_name: ZpoolName { id: 1bae97fb-c4f0-4d17-b41c-409574fb9e6e (zpool), kind: External }, kind: ExternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
8102024-09-06T23:36:25.355ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 195e4e82-1cc4-4f3b-a893-c1474bd26575, dataset_name: DatasetName { pool_name: ZpoolName { id: 1bae97fb-c4f0-4d17-b41c-409574fb9e6e (zpool), kind: External }, kind: ExternalDns }, responder: ... }
8112024-09-06T23:36:25.521ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2ac4364c-7933-4568-bbab-cf3f9ec0ff29, dataset_name: DatasetName { pool_name: ZpoolName { id: 652da97e-d553-4457-bc42-d10c0c455967 (zpool), kind: External }, kind: Clickhouse }, responder: ... }) file = sled-storage/src/manager.rs:451
8122024-09-06T23:36:25.521ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 2ac4364c-7933-4568-bbab-cf3f9ec0ff29, dataset_name: DatasetName { pool_name: ZpoolName { id: 652da97e-d553-4457-bc42-d10c0c455967 (zpool), kind: External }, kind: Clickhouse }, responder: ... }
8132024-09-06T23:36:25.615ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: e273cca1-6e0d-4509-842e-40a51b28e236, dataset_name: DatasetName { pool_name: ZpoolName { id: 652da97e-d553-4457-bc42-d10c0c455967 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:451
8142024-09-06T23:36:25.615ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: e273cca1-6e0d-4509-842e-40a51b28e236, dataset_name: DatasetName { pool_name: ZpoolName { id: 652da97e-d553-4457-bc42-d10c0c455967 (zpool), kind: External }, kind: Crucible }, responder: ... }
8152024-09-06T23:36:25.731ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2adfe4a1-4baf-4cb8-a3d2-e6907b84545a, dataset_name: DatasetName { pool_name: ZpoolName { id: 1bae97fb-c4f0-4d17-b41c-409574fb9e6e (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:451
8162024-09-06T23:36:25.731ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 2adfe4a1-4baf-4cb8-a3d2-e6907b84545a, dataset_name: DatasetName { pool_name: ZpoolName { id: 1bae97fb-c4f0-4d17-b41c-409574fb9e6e (zpool), kind: External }, kind: Crucible }, responder: ... }
8172024-09-06T23:36:25.851ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: dbf82130-678a-44d4-9dd9-8a8be286ccf6, dataset_name: DatasetName { pool_name: ZpoolName { id: eae5bedf-a363-4cf3-b83a-efcb44445301 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:451
8182024-09-06T23:36:25.851ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: dbf82130-678a-44d4-9dd9-8a8be286ccf6, dataset_name: DatasetName { pool_name: ZpoolName { id: eae5bedf-a363-4cf3-b83a-efcb44445301 (zpool), kind: External }, kind: Crucible }, responder: ... }
8192024-09-06T23:36:25.964ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 0a5539c1-7e26-4e50-b26a-60a137388ce1, dataset_name: DatasetName { pool_name: ZpoolName { id: ab9a060c-a3d5-48de-ba57-beaa10d18520 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:451
8202024-09-06T23:36:25.964ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 0a5539c1-7e26-4e50-b26a-60a137388ce1, dataset_name: DatasetName { pool_name: ZpoolName { id: ab9a060c-a3d5-48de-ba57-beaa10d18520 (zpool), kind: External }, kind: Crucible }, responder: ... }
8212024-09-06T23:36:26.077ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: cea6def4-eb13-4381-bb48-7596d86978d4, dataset_name: DatasetName { pool_name: ZpoolName { id: fd8d5a1e-9723-41c1-aa2b-460a6959d6c7 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:451
8222024-09-06T23:36:26.077ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: cea6def4-eb13-4381-bb48-7596d86978d4, dataset_name: DatasetName { pool_name: ZpoolName { id: fd8d5a1e-9723-41c1-aa2b-460a6959d6c7 (zpool), kind: External }, kind: Crucible }, responder: ... }
8232024-09-06T23:36:26.197ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ebce0088-396f-4300-91e9-19979170faa6, dataset_name: DatasetName { pool_name: ZpoolName { id: b79af253-dd33-4fe7-98c1-790c50e27394 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:451
8242024-09-06T23:36:26.197ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: ebce0088-396f-4300-91e9-19979170faa6, dataset_name: DatasetName { pool_name: ZpoolName { id: b79af253-dd33-4fe7-98c1-790c50e27394 (zpool), kind: External }, kind: Crucible }, responder: ... }
8252024-09-06T23:36:26.299ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: fa9373bc-3206-40d0-b421-9f54e256513c, dataset_name: DatasetName { pool_name: ZpoolName { id: cc3f0ccd-963c-4d0c-843d-e8810b230566 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:451
8262024-09-06T23:36:26.299ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: fa9373bc-3206-40d0-b421-9f54e256513c, dataset_name: DatasetName { pool_name: ZpoolName { id: cc3f0ccd-963c-4d0c-843d-e8810b230566 (zpool), kind: External }, kind: Crucible }, responder: ... }
8272024-09-06T23:36:26.403ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 6bc5dbb5-a884-4ae2-af09-70bc587bd76b, dataset_name: DatasetName { pool_name: ZpoolName { id: f8e03f3f-0403-4c51-926a-d5ce11d8408e (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:451
8282024-09-06T23:36:26.403ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: 6bc5dbb5-a884-4ae2-af09-70bc587bd76b, dataset_name: DatasetName { pool_name: ZpoolName { id: f8e03f3f-0403-4c51-926a-d5ce11d8408e (zpool), kind: External }, kind: Crucible }, responder: ... }
8292024-09-06T23:36:26.486ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c1d2ae9b-e921-4256-b95c-a29d6126fd66, dataset_name: DatasetName { pool_name: ZpoolName { id: 907237be-faec-4233-8c3e-1918ec810764 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:451
8302024-09-06T23:36:26.486ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1065 request = NewFilesystemRequest { dataset_id: c1d2ae9b-e921-4256-b95c-a29d6126fd66, dataset_name: DatasetName { pool_name: ZpoolName { id: 907237be-faec-4233-8c3e-1918ec810764 (zpool), kind: External }, kind: Crucible }, responder: ... }
8312024-09-06T23:36:26.569ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8322024-09-06T23:36:26.570ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/ca24db04-f37b-40b1-83c2-23476bec5a4a/config/omicron-zones.json file = common/src/ledger.rs:177
8332024-09-06T23:36:26.570ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/358fc814-3f2a-4b1a-aebb-894434d72d01/config/omicron-zones.json file = common/src/ledger.rs:177
8342024-09-06T23:36:26.570ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8352024-09-06T23:36:26.682ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
8362024-09-06T23:36:26.682ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8372024-09-06T23:36:26.697ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
8382024-09-06T23:36:26.697ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8392024-09-06T23:36:26.697ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8402024-09-06T23:36:26.697ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8412024-09-06T23:36:26.710ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
8422024-09-06T23:36:26.710ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8432024-09-06T23:36:26.722ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
8442024-09-06T23:36:26.733ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8452024-09-06T23:36:26.931ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
8462024-09-06T23:36:26.931ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8472024-09-06T23:36:26.931ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8482024-09-06T23:36:26.931ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8492024-09-06T23:36:26.931ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8502024-09-06T23:36:26.931ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8512024-09-06T23:36:26.931ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8522024-09-06T23:36:26.931ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8532024-09-06T23:36:26.942ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8542024-09-06T23:36:26.943ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_oximeter_bc144d23-6ff5-421e-a179-41beb5b46c0f file = illumos-utils/src/zone.rs:316
8552024-09-06T23:36:26.948ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de file = illumos-utils/src/zone.rs:316
8562024-09-06T23:36:26.953ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_072c2fe9-5a06-4344-a93a-991a46098bd3 file = illumos-utils/src/zone.rs:316
8572024-09-06T23:36:26.958ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_cea6def4-eb13-4381-bb48-7596d86978d4 file = illumos-utils/src/zone.rs:316
8582024-09-06T23:36:26.963ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_0a5539c1-7e26-4e50-b26a-60a137388ce1 file = illumos-utils/src/zone.rs:316
8592024-09-06T23:36:26.968ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_clickhouse_2ac4364c-7933-4568-bbab-cf3f9ec0ff29 file = illumos-utils/src/zone.rs:316
8602024-09-06T23:36:27.125ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_oximeter_bc144d23-6ff5-421e-a179-41beb5b46c0f file = illumos-utils/src/zone.rs:354
8612024-09-06T23:36:27.130ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de file = illumos-utils/src/zone.rs:354
8622024-09-06T23:36:27.135ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_072c2fe9-5a06-4344-a93a-991a46098bd3 file = illumos-utils/src/zone.rs:354
8632024-09-06T23:36:27.140ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_cea6def4-eb13-4381-bb48-7596d86978d4 file = illumos-utils/src/zone.rs:354
8642024-09-06T23:36:27.144ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_0a5539c1-7e26-4e50-b26a-60a137388ce1 file = illumos-utils/src/zone.rs:354
8652024-09-06T23:36:27.150ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_clickhouse_2ac4364c-7933-4568-bbab-cf3f9ec0ff29 file = illumos-utils/src/zone.rs:354
8662024-09-06T23:36:27.155ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_fa9373bc-3206-40d0-b421-9f54e256513c file = illumos-utils/src/zone.rs:316
8672024-09-06T23:36:27.160ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_6bc5dbb5-a884-4ae2-af09-70bc587bd76b file = illumos-utils/src/zone.rs:316
8682024-09-06T23:36:27.165ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_764a9699-9635-421f-88c6-7f14971487fe file = illumos-utils/src/zone.rs:316
8692024-09-06T23:36:27.170ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_2adfe4a1-4baf-4cb8-a3d2-e6907b84545a file = illumos-utils/src/zone.rs:316
8702024-09-06T23:36:27.175ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_dbf82130-678a-44d4-9dd9-8a8be286ccf6 file = illumos-utils/src/zone.rs:316
8712024-09-06T23:36:27.181ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_e273cca1-6e0d-4509-842e-40a51b28e236 file = illumos-utils/src/zone.rs:316
8722024-09-06T23:36:27.187ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_ebce0088-396f-4300-91e9-19979170faa6 file = illumos-utils/src/zone.rs:316
8732024-09-06T23:36:27.193ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_c1d2ae9b-e921-4256-b95c-a29d6126fd66 file = illumos-utils/src/zone.rs:316
8742024-09-06T23:36:27.236ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
8752024-09-06T23:36:27.236ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_fa9373bc-3206-40d0-b421-9f54e256513c file = illumos-utils/src/zone.rs:354
8762024-09-06T23:36:27.241ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_6bc5dbb5-a884-4ae2-af09-70bc587bd76b file = illumos-utils/src/zone.rs:354
8772024-09-06T23:36:27.246ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_764a9699-9635-421f-88c6-7f14971487fe file = illumos-utils/src/zone.rs:354
8782024-09-06T23:36:27.250ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_2adfe4a1-4baf-4cb8-a3d2-e6907b84545a file = illumos-utils/src/zone.rs:354
8792024-09-06T23:36:27.255ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_dbf82130-678a-44d4-9dd9-8a8be286ccf6 file = illumos-utils/src/zone.rs:354
8802024-09-06T23:36:27.259ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_e273cca1-6e0d-4509-842e-40a51b28e236 file = illumos-utils/src/zone.rs:354
8812024-09-06T23:36:27.264ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_ebce0088-396f-4300-91e9-19979170faa6 file = illumos-utils/src/zone.rs:354
8822024-09-06T23:36:27.269ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_c1d2ae9b-e921-4256-b95c-a29d6126fd66 file = illumos-utils/src/zone.rs:354
8832024-09-06T23:36:27.323ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
8842024-09-06T23:36:27.367ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
8852024-09-06T23:36:27.410ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
8862024-09-06T23:36:27.460ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
8872024-09-06T23:36:27.465ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
8882024-09-06T23:36:27.465ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
8892024-09-06T23:36:27.590ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte1", ip: 172.30.2.5, mac: MacAddr6([168, 64, 37, 255, 248, 203]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
8902024-09-06T23:36:27.590ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1234 zone_type = nexus
8912024-09-06T23:36:27.593ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
8922024-09-06T23:36:27.593ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
8932024-09-06T23:36:27.643ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte2", ip: 172.30.2.6, mac: MacAddr6([168, 64, 37, 255, 218, 98]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
8942024-09-06T23:36:27.643ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1234 zone_type = nexus
8952024-09-06T23:36:27.645ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
8962024-09-06T23:36:27.645ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
8972024-09-06T23:36:27.695ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte3", ip: 172.30.1.5, mac: MacAddr6([168, 64, 37, 255, 179, 238]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.1.1 } }) }
8982024-09-06T23:36:27.695ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1234 zone_type = external_dns
8992024-09-06T23:36:27.696ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
9002024-09-06T23:36:27.696ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
9012024-09-06T23:36:27.746ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte4", ip: 172.30.1.6, mac: MacAddr6([168, 64, 37, 255, 212, 58]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.1.1 } }) }
9022024-09-06T23:36:27.746ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1234 zone_type = external_dns
9032024-09-06T23:36:27.747ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
9042024-09-06T23:36:27.747ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
9052024-09-06T23:36:27.799ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte5", ip: 172.30.2.7, mac: MacAddr6([168, 64, 37, 255, 170, 148]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
9062024-09-06T23:36:27.799ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1234 zone_type = nexus
9072024-09-06T23:36:27.799ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.166 file = clients/dpd-client/src/lib.rs:83
9082024-09-06T23:36:27.799ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.166 file = clients/dpd-client/src/lib.rs:90
9092024-09-06T23:36:27.799ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.165 file = clients/dpd-client/src/lib.rs:83
9102024-09-06T23:36:27.799ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.165 file = clients/dpd-client/src/lib.rs:90
9112024-09-06T23:36:27.800ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.167 file = clients/dpd-client/src/lib.rs:83
9122024-09-06T23:36:27.800ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.167 file = clients/dpd-client/src/lib.rs:90
9132024-09-06T23:36:27.800ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.165 file = clients/dpd-client/src/lib.rs:118
9142024-09-06T23:36:27.801ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
9152024-09-06T23:36:27.801ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.168 file = clients/dpd-client/src/lib.rs:83
9162024-09-06T23:36:27.801ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.168 file = clients/dpd-client/src/lib.rs:90
9172024-09-06T23:36:27.801ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.166 file = clients/dpd-client/src/lib.rs:118
9182024-09-06T23:36:27.803ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.167 file = clients/dpd-client/src/lib.rs:118
9192024-09-06T23:36:27.805ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
9202024-09-06T23:36:27.805ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
9212024-09-06T23:36:27.805ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.164 file = clients/dpd-client/src/lib.rs:83
9222024-09-06T23:36:27.805ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.164 file = clients/dpd-client/src/lib.rs:90
9232024-09-06T23:36:27.840ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.168 file = clients/dpd-client/src/lib.rs:118
9242024-09-06T23:36:27.919ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.164 file = clients/dpd-client/src/lib.rs:118
9252024-09-06T23:36:27.922ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
9262024-09-06T23:36:27.922ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
9272024-09-06T23:36:27.923ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_external_dns_195e4e82-1cc4-4f3b-a893-c1474bd26575 file = illumos-utils/src/zone.rs:316
9282024-09-06T23:36:27.977ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_9ad0f243-aeb1-455d-97ad-7dc9ca9c417c file = illumos-utils/src/zone.rs:316
9292024-09-06T23:36:27.992ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_6c363cb4-229c-47a8-9d81-abd1714293f2 file = illumos-utils/src/zone.rs:316
9302024-09-06T23:36:28.052ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_external_dns_195e4e82-1cc4-4f3b-a893-c1474bd26575 file = illumos-utils/src/zone.rs:354
9312024-09-06T23:36:28.062ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_cb51aad1-b6ab-44e1-8492-82c028d74c7f file = illumos-utils/src/zone.rs:316
9322024-09-06T23:36:28.073ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_9ad0f243-aeb1-455d-97ad-7dc9ca9c417c file = illumos-utils/src/zone.rs:354
9332024-09-06T23:36:28.082ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_6c363cb4-229c-47a8-9d81-abd1714293f2 file = illumos-utils/src/zone.rs:354
9342024-09-06T23:36:28.092ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_external_dns_5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6 file = illumos-utils/src/zone.rs:316
9352024-09-06T23:36:28.104ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_cb51aad1-b6ab-44e1-8492-82c028d74c7f file = illumos-utils/src/zone.rs:354
9362024-09-06T23:36:28.131ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_external_dns_5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6 file = illumos-utils/src/zone.rs:354
9372024-09-06T23:36:28.667ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService11'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::10'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::10'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9382024-09-06T23:36:28.667ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de
9392024-09-06T23:36:28.765ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_072c2fe9-5a06-4344-a93a-991a46098bd3: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService12'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::f'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::f'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9402024-09-06T23:36:28.766ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_072c2fe9-5a06-4344-a93a-991a46098bd3 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_072c2fe9-5a06-4344-a93a-991a46098bd3
9412024-09-06T23:36:28.775ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_0a5539c1-7e26-4e50-b26a-60a137388ce1: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService14'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::15'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_ab9a060c-a3d5-48de-ba57-beaa10d18520/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::15'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='e405531c-a582-4aa3-96c7-35a9353cf7f5'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9422024-09-06T23:36:28.775ZINFOSledAgent (ServiceManager): Booting oxz_crucible_0a5539c1-7e26-4e50-b26a-60a137388ce1 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_0a5539c1-7e26-4e50-b26a-60a137388ce1
9432024-09-06T23:36:28.840ZINFOSledAgent (ServiceManager): Profile for oxz_oximeter_bc144d23-6ff5-421e-a179-41beb5b46c0f: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService10'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::d'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/oximeter"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="id" value='bc144d23-6ff5-421e-a179-41beb5b46c0f'/> <propval type="astring" name="address" value='[fd00:1122:3344:101::d]:12223'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9442024-09-06T23:36:28.840ZINFOSledAgent (ServiceManager): Booting oxz_oximeter_bc144d23-6ff5-421e-a179-41beb5b46c0f zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_oximeter_bc144d23-6ff5-421e-a179-41beb5b46c0f
9452024-09-06T23:36:28.865ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_fa9373bc-3206-40d0-b421-9f54e256513c: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService16'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::18'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_cc3f0ccd-963c-4d0c-843d-e8810b230566/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::18'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='93a0ad8d-e4c2-4093-af2a-76710b2d3c8f'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9462024-09-06T23:36:28.865ZINFOSledAgent (ServiceManager): Booting oxz_crucible_fa9373bc-3206-40d0-b421-9f54e256513c zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_fa9373bc-3206-40d0-b421-9f54e256513c
9472024-09-06T23:36:28.880ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_2adfe4a1-4baf-4cb8-a3d2-e6907b84545a: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService19'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::13'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_1bae97fb-c4f0-4d17-b41c-409574fb9e6e/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::13'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='c3a8fa2b-32a3-45f6-8f63-dd8f84867df1'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9482024-09-06T23:36:28.881ZINFOSledAgent (ServiceManager): Booting oxz_crucible_2adfe4a1-4baf-4cb8-a3d2-e6907b84545a zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_2adfe4a1-4baf-4cb8-a3d2-e6907b84545a
9492024-09-06T23:36:28.890ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_ebce0088-396f-4300-91e9-19979170faa6: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService22'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::17'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_b79af253-dd33-4fe7-98c1-790c50e27394/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::17'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='73009da3-46f5-47db-91a6-f451840c7427'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9502024-09-06T23:36:28.890ZINFOSledAgent (ServiceManager): Booting oxz_crucible_ebce0088-396f-4300-91e9-19979170faa6 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_ebce0088-396f-4300-91e9-19979170faa6
9512024-09-06T23:36:28.945ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_cea6def4-eb13-4381-bb48-7596d86978d4: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService13'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::16'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_fd8d5a1e-9723-41c1-aa2b-460a6959d6c7/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::16'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='07ae874d-c98c-4850-9928-7b31e1807044'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9522024-09-06T23:36:28.945ZINFOSledAgent (ServiceManager): Booting oxz_crucible_cea6def4-eb13-4381-bb48-7596d86978d4 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_cea6def4-eb13-4381-bb48-7596d86978d4
9532024-09-06T23:36:29.208ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2adfe4a1-4baf-4cb8-a3d2-e6907b84545a") failed: Property not found. retry in 26.803559ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2adfe4a1-4baf-4cb8-a3d2-e6907b84545a
9542024-09-06T23:36:29.215ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_bc144d23-6ff5-421e-a179-41beb5b46c0f") failed: Property not found. retry in 65.726734ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_bc144d23-6ff5-421e-a179-41beb5b46c0f
9552024-09-06T23:36:29.221ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de") failed: Property not found. retry in 28.603225ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de
9562024-09-06T23:36:29.228ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_072c2fe9-5a06-4344-a93a-991a46098bd3") failed: Property not found. retry in 31.119502ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_072c2fe9-5a06-4344-a93a-991a46098bd3
9572024-09-06T23:36:29.236ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cea6def4-eb13-4381-bb48-7596d86978d4") failed: Property not found. retry in 56.845243ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cea6def4-eb13-4381-bb48-7596d86978d4
9582024-09-06T23:36:29.242ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0a5539c1-7e26-4e50-b26a-60a137388ce1") failed: Property not found. retry in 66.030097ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0a5539c1-7e26-4e50-b26a-60a137388ce1
9592024-09-06T23:36:29.249ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fa9373bc-3206-40d0-b421-9f54e256513c") failed: Property not found. retry in 57.383556ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fa9373bc-3206-40d0-b421-9f54e256513c
9602024-09-06T23:36:29.255ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ebce0088-396f-4300-91e9-19979170faa6") failed: Property not found. retry in 41.447095ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ebce0088-396f-4300-91e9-19979170faa6
9612024-09-06T23:36:29.262ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de") failed: Property not found. retry in 124.624899ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de
9622024-09-06T23:36:29.268ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2adfe4a1-4baf-4cb8-a3d2-e6907b84545a") failed: Property not found. retry in 60.558159ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2adfe4a1-4baf-4cb8-a3d2-e6907b84545a
9632024-09-06T23:36:29.307ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_072c2fe9-5a06-4344-a93a-991a46098bd3") failed: Property not found. retry in 143.431193ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_072c2fe9-5a06-4344-a93a-991a46098bd3
9642024-09-06T23:36:29.362ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_bc144d23-6ff5-421e-a179-41beb5b46c0f") failed: Property not found. retry in 73.768529ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_bc144d23-6ff5-421e-a179-41beb5b46c0f
9652024-09-06T23:36:29.484ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cea6def4-eb13-4381-bb48-7596d86978d4") failed: Property not found. retry in 94.222462ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cea6def4-eb13-4381-bb48-7596d86978d4
9662024-09-06T23:36:29.721ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0a5539c1-7e26-4e50-b26a-60a137388ce1") failed: Property not found. retry in 113.903776ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0a5539c1-7e26-4e50-b26a-60a137388ce1
9672024-09-06T23:36:29.761ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fa9373bc-3206-40d0-b421-9f54e256513c") failed: Property not found. retry in 113.820246ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fa9373bc-3206-40d0-b421-9f54e256513c
9682024-09-06T23:36:29.844ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2adfe4a1-4baf-4cb8-a3d2-e6907b84545a") failed: Property not found. retry in 296.662097ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2adfe4a1-4baf-4cb8-a3d2-e6907b84545a
9692024-09-06T23:36:30.227ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ebce0088-396f-4300-91e9-19979170faa6") failed: Property not found. retry in 88.454869ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ebce0088-396f-4300-91e9-19979170faa6
9702024-09-06T23:36:30.324ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_bc144d23-6ff5-421e-a179-41beb5b46c0f") failed: Property not found. retry in 228.251386ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_bc144d23-6ff5-421e-a179-41beb5b46c0f
9712024-09-06T23:36:30.346ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de") failed: Property not found. retry in 260.205925ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de
9722024-09-06T23:36:30.400ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_072c2fe9-5a06-4344-a93a-991a46098bd3") failed: Property not found. retry in 285.514579ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_072c2fe9-5a06-4344-a93a-991a46098bd3
9732024-09-06T23:36:30.458ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cea6def4-eb13-4381-bb48-7596d86978d4") failed: Property not found. retry in 228.588148ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cea6def4-eb13-4381-bb48-7596d86978d4
9742024-09-06T23:36:30.484ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0a5539c1-7e26-4e50-b26a-60a137388ce1") failed: Property not found. retry in 153.922793ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0a5539c1-7e26-4e50-b26a-60a137388ce1
9752024-09-06T23:36:30.504ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fa9373bc-3206-40d0-b421-9f54e256513c") failed: Property not found. retry in 271.980681ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fa9373bc-3206-40d0-b421-9f54e256513c
9762024-09-06T23:36:30.524ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2adfe4a1-4baf-4cb8-a3d2-e6907b84545a") failed: Property not found. retry in 261.284819ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2adfe4a1-4baf-4cb8-a3d2-e6907b84545a
9772024-09-06T23:36:30.556ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ebce0088-396f-4300-91e9-19979170faa6") failed: Property not found. retry in 140.513722ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ebce0088-396f-4300-91e9-19979170faa6
9782024-09-06T23:36:30.582ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_bc144d23-6ff5-421e-a179-41beb5b46c0f") failed: Property not found. retry in 289.509615ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_bc144d23-6ff5-421e-a179-41beb5b46c0f
9792024-09-06T23:36:30.670ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de") failed: Property not found. retry in 465.011409ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de
9802024-09-06T23:36:30.684ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0a5539c1-7e26-4e50-b26a-60a137388ce1") failed: Property not found. retry in 477.651217ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0a5539c1-7e26-4e50-b26a-60a137388ce1
9812024-09-06T23:36:30.697ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_072c2fe9-5a06-4344-a93a-991a46098bd3") failed: Property not found. retry in 584.932776ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_072c2fe9-5a06-4344-a93a-991a46098bd3
9822024-09-06T23:36:30.712ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cea6def4-eb13-4381-bb48-7596d86978d4") failed: Property not found. retry in 230.347477ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cea6def4-eb13-4381-bb48-7596d86978d4
9832024-09-06T23:36:30.725ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ebce0088-396f-4300-91e9-19979170faa6") failed: Property not found. retry in 477.627636ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ebce0088-396f-4300-91e9-19979170faa6
9842024-09-06T23:36:30.791ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fa9373bc-3206-40d0-b421-9f54e256513c") failed: Property not found. retry in 213.476678ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fa9373bc-3206-40d0-b421-9f54e256513c
9852024-09-06T23:36:30.805ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2adfe4a1-4baf-4cb8-a3d2-e6907b84545a") failed: Property not found. retry in 901.334914ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2adfe4a1-4baf-4cb8-a3d2-e6907b84545a
9862024-09-06T23:36:30.886ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_bc144d23-6ff5-421e-a179-41beb5b46c0f") failed: Property not found. retry in 885.374453ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_bc144d23-6ff5-421e-a179-41beb5b46c0f
9872024-09-06T23:36:30.953ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cea6def4-eb13-4381-bb48-7596d86978d4") failed: Property not found. retry in 927.273174ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cea6def4-eb13-4381-bb48-7596d86978d4
9882024-09-06T23:36:31.018ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fa9373bc-3206-40d0-b421-9f54e256513c") failed: Property not found. retry in 884.756685ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fa9373bc-3206-40d0-b421-9f54e256513c
9892024-09-06T23:36:31.152ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de") failed: Property not found. retry in 729.584782ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de
9902024-09-06T23:36:31.173ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0a5539c1-7e26-4e50-b26a-60a137388ce1") failed: Property not found. retry in 675.327299ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0a5539c1-7e26-4e50-b26a-60a137388ce1
9912024-09-06T23:36:31.215ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ebce0088-396f-4300-91e9-19979170faa6") failed: Property not found. retry in 1.198746505s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ebce0088-396f-4300-91e9-19979170faa6
9922024-09-06T23:36:31.295ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_072c2fe9-5a06-4344-a93a-991a46098bd3") failed: Property not found. retry in 1.017038342s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_072c2fe9-5a06-4344-a93a-991a46098bd3
9932024-09-06T23:36:31.608ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_6bc5dbb5-a884-4ae2-af09-70bc587bd76b: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService17'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::19'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_f8e03f3f-0403-4c51-926a-d5ce11d8408e/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::19'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='4b75be5c-b971-4e22-8d20-9026debff29a'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9942024-09-06T23:36:31.608ZINFOSledAgent (ServiceManager): Booting oxz_crucible_6bc5dbb5-a884-4ae2-af09-70bc587bd76b zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_6bc5dbb5-a884-4ae2-af09-70bc587bd76b
9952024-09-06T23:36:31.627ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_e273cca1-6e0d-4509-842e-40a51b28e236: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService21'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::12'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_652da97e-d553-4457-bc42-d10c0c455967/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::12'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='915a55f1-dfaf-4bb2-8fce-8b8f3209dc11'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9962024-09-06T23:36:31.627ZINFOSledAgent (ServiceManager): Booting oxz_crucible_e273cca1-6e0d-4509-842e-40a51b28e236 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_e273cca1-6e0d-4509-842e-40a51b28e236
9972024-09-06T23:36:31.664ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_dbf82130-678a-44d4-9dd9-8a8be286ccf6: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService20'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::14'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_eae5bedf-a363-4cf3-b83a-efcb44445301/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::14'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='95394992-8fd7-4fc1-ab3f-0165bc6888ba'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9982024-09-06T23:36:31.664ZINFOSledAgent (ServiceManager): Booting oxz_crucible_dbf82130-678a-44d4-9dd9-8a8be286ccf6 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_dbf82130-678a-44d4-9dd9-8a8be286ccf6
9992024-09-06T23:36:31.721ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2adfe4a1-4baf-4cb8-a3d2-e6907b84545a") failed: Property not found. retry in 1.3348099s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2adfe4a1-4baf-4cb8-a3d2-e6907b84545a
10002024-09-06T23:36:31.785ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_bc144d23-6ff5-421e-a179-41beb5b46c0f") failed: Property not found. retry in 1.326095016s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_bc144d23-6ff5-421e-a179-41beb5b46c0f
10012024-09-06T23:36:31.837ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e273cca1-6e0d-4509-842e-40a51b28e236") failed: Property not found. retry in 64.352372ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e273cca1-6e0d-4509-842e-40a51b28e236
10022024-09-06T23:36:31.848ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6bc5dbb5-a884-4ae2-af09-70bc587bd76b") failed: Property not found. retry in 35.836131ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6bc5dbb5-a884-4ae2-af09-70bc587bd76b
10032024-09-06T23:36:31.860ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0a5539c1-7e26-4e50-b26a-60a137388ce1") failed: Property not found. retry in 1.299472589s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0a5539c1-7e26-4e50-b26a-60a137388ce1
10042024-09-06T23:36:31.870ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_dbf82130-678a-44d4-9dd9-8a8be286ccf6") failed: Property not found. retry in 72.882524ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_dbf82130-678a-44d4-9dd9-8a8be286ccf6
10052024-09-06T23:36:31.894ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de") failed: Property not found. retry in 851.025368ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de
10062024-09-06T23:36:31.905ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cea6def4-eb13-4381-bb48-7596d86978d4") failed: Property not found. retry in 1.098598461s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cea6def4-eb13-4381-bb48-7596d86978d4
10072024-09-06T23:36:31.914ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fa9373bc-3206-40d0-b421-9f54e256513c") failed: Property not found. retry in 1.390120884s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fa9373bc-3206-40d0-b421-9f54e256513c
10082024-09-06T23:36:31.924ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6bc5dbb5-a884-4ae2-af09-70bc587bd76b") failed: Property not found. retry in 115.560111ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6bc5dbb5-a884-4ae2-af09-70bc587bd76b
10092024-09-06T23:36:31.933ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e273cca1-6e0d-4509-842e-40a51b28e236") failed: Property not found. retry in 81.775775ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e273cca1-6e0d-4509-842e-40a51b28e236
10102024-09-06T23:36:31.954ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_dbf82130-678a-44d4-9dd9-8a8be286ccf6") failed: Property not found. retry in 81.806561ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_dbf82130-678a-44d4-9dd9-8a8be286ccf6
10112024-09-06T23:36:32.026ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e273cca1-6e0d-4509-842e-40a51b28e236") failed: Property not found. retry in 263.166563ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e273cca1-6e0d-4509-842e-40a51b28e236
10122024-09-06T23:36:32.047ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_dbf82130-678a-44d4-9dd9-8a8be286ccf6") failed: Property not found. retry in 142.450024ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_dbf82130-678a-44d4-9dd9-8a8be286ccf6
10132024-09-06T23:36:32.060ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6bc5dbb5-a884-4ae2-af09-70bc587bd76b") failed: Property not found. retry in 278.958423ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6bc5dbb5-a884-4ae2-af09-70bc587bd76b
10142024-09-06T23:36:32.293ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_dbf82130-678a-44d4-9dd9-8a8be286ccf6") failed: Property not found. retry in 413.364858ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_dbf82130-678a-44d4-9dd9-8a8be286ccf6
10152024-09-06T23:36:32.313ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e273cca1-6e0d-4509-842e-40a51b28e236") failed: Property not found. retry in 567.176497ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e273cca1-6e0d-4509-842e-40a51b28e236
10162024-09-06T23:36:32.391ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_072c2fe9-5a06-4344-a93a-991a46098bd3") failed: Property not found. retry in 1.019996587s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_072c2fe9-5a06-4344-a93a-991a46098bd3
10172024-09-06T23:36:32.415ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6bc5dbb5-a884-4ae2-af09-70bc587bd76b") failed: Property not found. retry in 334.280579ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6bc5dbb5-a884-4ae2-af09-70bc587bd76b
10182024-09-06T23:36:32.440ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ebce0088-396f-4300-91e9-19979170faa6") failed: Property not found. retry in 735.563756ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ebce0088-396f-4300-91e9-19979170faa6
10192024-09-06T23:36:32.728ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_dbf82130-678a-44d4-9dd9-8a8be286ccf6") failed: Property not found. retry in 434.379574ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_dbf82130-678a-44d4-9dd9-8a8be286ccf6
10202024-09-06T23:36:32.829ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de") failed: Property not found. retry in 720.464348ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de
10212024-09-06T23:36:32.898ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6bc5dbb5-a884-4ae2-af09-70bc587bd76b") failed: Property not found. retry in 841.188291ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6bc5dbb5-a884-4ae2-af09-70bc587bd76b
10222024-09-06T23:36:32.953ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e273cca1-6e0d-4509-842e-40a51b28e236") failed: Property not found. retry in 843.480655ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e273cca1-6e0d-4509-842e-40a51b28e236
10232024-09-06T23:36:33.021ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cea6def4-eb13-4381-bb48-7596d86978d4") failed: Property not found. retry in 670.715759ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cea6def4-eb13-4381-bb48-7596d86978d4
10242024-09-06T23:36:33.073ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2adfe4a1-4baf-4cb8-a3d2-e6907b84545a") failed: Property not found. retry in 543.505108ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2adfe4a1-4baf-4cb8-a3d2-e6907b84545a
10252024-09-06T23:36:33.125ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_bc144d23-6ff5-421e-a179-41beb5b46c0f") failed: Property not found. retry in 763.242091ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_bc144d23-6ff5-421e-a179-41beb5b46c0f
10262024-09-06T23:36:33.174ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0a5539c1-7e26-4e50-b26a-60a137388ce1") failed: Property not found. retry in 1.02210566s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0a5539c1-7e26-4e50-b26a-60a137388ce1
10272024-09-06T23:36:33.184ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_dbf82130-678a-44d4-9dd9-8a8be286ccf6") failed: Property not found. retry in 1.208386386s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_dbf82130-678a-44d4-9dd9-8a8be286ccf6
10282024-09-06T23:36:33.193ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ebce0088-396f-4300-91e9-19979170faa6") failed: Property not found. retry in 1.347100034s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ebce0088-396f-4300-91e9-19979170faa6
10292024-09-06T23:36:33.318ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fa9373bc-3206-40d0-b421-9f54e256513c") failed: Property not found. retry in 702.633276ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fa9373bc-3206-40d0-b421-9f54e256513c
10302024-09-06T23:36:33.424ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_072c2fe9-5a06-4344-a93a-991a46098bd3") failed: Property not found. retry in 1.170219802s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_072c2fe9-5a06-4344-a93a-991a46098bd3
10312024-09-06T23:36:33.591ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de") failed: Property not found. retry in 939.181806ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de
10322024-09-06T23:36:33.645ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2adfe4a1-4baf-4cb8-a3d2-e6907b84545a") failed: Property not found. retry in 1.42038982s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2adfe4a1-4baf-4cb8-a3d2-e6907b84545a
10332024-09-06T23:36:33.713ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cea6def4-eb13-4381-bb48-7596d86978d4") failed: Property not found. retry in 907.909246ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cea6def4-eb13-4381-bb48-7596d86978d4
10342024-09-06T23:36:33.763ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6bc5dbb5-a884-4ae2-af09-70bc587bd76b") failed: Property not found. retry in 1.346610558s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6bc5dbb5-a884-4ae2-af09-70bc587bd76b
10352024-09-06T23:36:33.816ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e273cca1-6e0d-4509-842e-40a51b28e236") failed: Property not found. retry in 847.60374ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e273cca1-6e0d-4509-842e-40a51b28e236
10362024-09-06T23:36:33.913ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_bc144d23-6ff5-421e-a179-41beb5b46c0f") failed: Property not found. retry in 1.418144029s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_bc144d23-6ff5-421e-a179-41beb5b46c0f
10372024-09-06T23:36:34.078ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fa9373bc-3206-40d0-b421-9f54e256513c") failed: Property not found. retry in 1.498242239s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fa9373bc-3206-40d0-b421-9f54e256513c
10382024-09-06T23:36:34.079ZINFOSledAgent (ServiceManager): Profile for oxz_external_dns_195e4e82-1cc4-4f3b-a893-c1474bd26575: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService24'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::9'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte4'/> <propval type="astring" name="gateway" value='172.30.1.1'/> <propval type="astring" name="ip" value='172.30.1.6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/external_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::9]:5353'/> <propval type="astring" name="dns_address" value='172.30.1.6:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10392024-09-06T23:36:34.079ZINFOSledAgent (ServiceManager): Booting oxz_external_dns_195e4e82-1cc4-4f3b-a893-c1474bd26575 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_external_dns_195e4e82-1cc4-4f3b-a893-c1474bd26575
10402024-09-06T23:36:34.114ZINFOSledAgent (ServiceManager): Profile for oxz_clickhouse_2ac4364c-7933-4568-bbab-cf3f9ec0ff29: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService15'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::e'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/clickhouse"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::e'/> <propval type="astring" name="listen_port" value='8123'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/clickhouse-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::e]:8888'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10412024-09-06T23:36:34.114ZINFOSledAgent (ServiceManager): Booting oxz_clickhouse_2ac4364c-7933-4568-bbab-cf3f9ec0ff29 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_clickhouse_2ac4364c-7933-4568-bbab-cf3f9ec0ff29
10422024-09-06T23:36:34.218ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0a5539c1-7e26-4e50-b26a-60a137388ce1") failed: Property not found. retry in 1.02179961s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0a5539c1-7e26-4e50-b26a-60a137388ce1
10432024-09-06T23:36:34.410ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_dbf82130-678a-44d4-9dd9-8a8be286ccf6") failed: Property not found. retry in 1.489765195s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_dbf82130-678a-44d4-9dd9-8a8be286ccf6
10442024-09-06T23:36:34.501ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2ac4364c-7933-4568-bbab-cf3f9ec0ff29") failed: Property not found. retry in 73.703435ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2ac4364c-7933-4568-bbab-cf3f9ec0ff29
10452024-09-06T23:36:34.524ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_195e4e82-1cc4-4f3b-a893-c1474bd26575") failed: Property not found. retry in 61.852844ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_195e4e82-1cc4-4f3b-a893-c1474bd26575
10462024-09-06T23:36:34.545ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de") failed: Property not found. retry in 500.956452ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de
10472024-09-06T23:36:34.559ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ebce0088-396f-4300-91e9-19979170faa6") failed: Property not found. retry in 1.237851832s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ebce0088-396f-4300-91e9-19979170faa6
10482024-09-06T23:36:34.589ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2ac4364c-7933-4568-bbab-cf3f9ec0ff29") failed: Property not found. retry in 78.032415ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2ac4364c-7933-4568-bbab-cf3f9ec0ff29
10492024-09-06T23:36:34.600ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_195e4e82-1cc4-4f3b-a893-c1474bd26575") failed: Property not found. retry in 86.137194ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_195e4e82-1cc4-4f3b-a893-c1474bd26575
10502024-09-06T23:36:34.613ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_072c2fe9-5a06-4344-a93a-991a46098bd3") failed: Property not found. retry in 877.450277ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_072c2fe9-5a06-4344-a93a-991a46098bd3
10512024-09-06T23:36:34.634ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cea6def4-eb13-4381-bb48-7596d86978d4") failed: Property not found. retry in 1.253793926s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cea6def4-eb13-4381-bb48-7596d86978d4
10522024-09-06T23:36:34.677ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e273cca1-6e0d-4509-842e-40a51b28e236") failed: Property not found. retry in 1.000275645s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e273cca1-6e0d-4509-842e-40a51b28e236
10532024-09-06T23:36:34.688ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2ac4364c-7933-4568-bbab-cf3f9ec0ff29") failed: Property not found. retry in 226.522098ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2ac4364c-7933-4568-bbab-cf3f9ec0ff29
10542024-09-06T23:36:34.699ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_195e4e82-1cc4-4f3b-a893-c1474bd26575") failed: Property not found. retry in 293.693898ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_195e4e82-1cc4-4f3b-a893-c1474bd26575
10552024-09-06T23:36:34.813ZINFOSledAgent (ServiceManager): Profile for oxz_external_dns_5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService28'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::8'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte3'/> <propval type="astring" name="gateway" value='172.30.1.1'/> <propval type="astring" name="ip" value='172.30.1.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/external_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::8]:5353'/> <propval type="astring" name="dns_address" value='172.30.1.5:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10562024-09-06T23:36:34.813ZINFOSledAgent (ServiceManager): Booting oxz_external_dns_5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_external_dns_5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6
10572024-09-06T23:36:34.931ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2ac4364c-7933-4568-bbab-cf3f9ec0ff29") failed: Property not found. retry in 557.360773ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2ac4364c-7933-4568-bbab-cf3f9ec0ff29
10582024-09-06T23:36:35.008ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_195e4e82-1cc4-4f3b-a893-c1474bd26575") failed: Property not found. retry in 590.659188ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_195e4e82-1cc4-4f3b-a893-c1474bd26575
10592024-09-06T23:36:35.009ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_9ad0f243-aeb1-455d-97ad-7dc9ca9c417c: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService25'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::a'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte1'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10602024-09-06T23:36:35.009ZINFOSledAgent (ServiceManager): Booting oxz_nexus_9ad0f243-aeb1-455d-97ad-7dc9ca9c417c zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_9ad0f243-aeb1-455d-97ad-7dc9ca9c417c
10612024-09-06T23:36:35.064ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de") failed: Property not found. retry in 1.303961694s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de
10622024-09-06T23:36:35.078ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2adfe4a1-4baf-4cb8-a3d2-e6907b84545a") failed: Property not found. retry in 633.175633ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2adfe4a1-4baf-4cb8-a3d2-e6907b84545a
10632024-09-06T23:36:35.110ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6") failed: Property not found. retry in 25.865786ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6
10642024-09-06T23:36:35.120ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6bc5dbb5-a884-4ae2-af09-70bc587bd76b") failed: Property not found. retry in 1.30740874s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6bc5dbb5-a884-4ae2-af09-70bc587bd76b
10652024-09-06T23:36:35.144ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6") failed: Property not found. retry in 51.96521ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6
10662024-09-06T23:36:35.207ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6") failed: Property not found. retry in 172.452826ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6
10672024-09-06T23:36:35.253ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0a5539c1-7e26-4e50-b26a-60a137388ce1") failed: Property not found. retry in 1.166088234s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0a5539c1-7e26-4e50-b26a-60a137388ce1
10682024-09-06T23:36:35.295ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_9ad0f243-aeb1-455d-97ad-7dc9ca9c417c") failed: Property not found. retry in 57.967031ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_9ad0f243-aeb1-455d-97ad-7dc9ca9c417c
10692024-09-06T23:36:35.344ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_bc144d23-6ff5-421e-a179-41beb5b46c0f") failed: Property not found. retry in 1.247337355s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_bc144d23-6ff5-421e-a179-41beb5b46c0f
10702024-09-06T23:36:35.364ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_9ad0f243-aeb1-455d-97ad-7dc9ca9c417c") failed: Property not found. retry in 72.728476ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_9ad0f243-aeb1-455d-97ad-7dc9ca9c417c
10712024-09-06T23:36:35.394ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6") failed: Property not found. retry in 311.303451ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6
10722024-09-06T23:36:35.519ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_9ad0f243-aeb1-455d-97ad-7dc9ca9c417c") failed: Property not found. retry in 269.090467ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_9ad0f243-aeb1-455d-97ad-7dc9ca9c417c
10732024-09-06T23:36:35.566ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_072c2fe9-5a06-4344-a93a-991a46098bd3") failed: Property not found. retry in 744.60406ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_072c2fe9-5a06-4344-a93a-991a46098bd3
10742024-09-06T23:36:35.583ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2ac4364c-7933-4568-bbab-cf3f9ec0ff29") failed: Property not found. retry in 455.115103ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2ac4364c-7933-4568-bbab-cf3f9ec0ff29
10752024-09-06T23:36:35.599ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fa9373bc-3206-40d0-b421-9f54e256513c") failed: Property not found. retry in 1.034820996s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fa9373bc-3206-40d0-b421-9f54e256513c
10762024-09-06T23:36:35.625ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_195e4e82-1cc4-4f3b-a893-c1474bd26575") failed: Property not found. retry in 989.382398ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_195e4e82-1cc4-4f3b-a893-c1474bd26575
10772024-09-06T23:36:35.700ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e273cca1-6e0d-4509-842e-40a51b28e236") failed: Property not found. retry in 699.310674ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e273cca1-6e0d-4509-842e-40a51b28e236
10782024-09-06T23:36:35.727ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6") failed: Property not found. retry in 424.190944ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6
10792024-09-06T23:36:35.757ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2adfe4a1-4baf-4cb8-a3d2-e6907b84545a") failed: Property not found. retry in 1.322602775s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2adfe4a1-4baf-4cb8-a3d2-e6907b84545a
10802024-09-06T23:36:35.757ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_764a9699-9635-421f-88c6-7f14971487fe: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService18'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::11'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::11'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10812024-09-06T23:36:35.758ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_764a9699-9635-421f-88c6-7f14971487fe zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_764a9699-9635-421f-88c6-7f14971487fe
10822024-09-06T23:36:35.820ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_9ad0f243-aeb1-455d-97ad-7dc9ca9c417c") failed: Property not found. retry in 499.101643ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_9ad0f243-aeb1-455d-97ad-7dc9ca9c417c
10832024-09-06T23:36:35.988ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ebce0088-396f-4300-91e9-19979170faa6") failed: Property not found. retry in 698.897794ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ebce0088-396f-4300-91e9-19979170faa6
10842024-09-06T23:36:36.056ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cea6def4-eb13-4381-bb48-7596d86978d4") failed: Property not found. retry in 1.310916957s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cea6def4-eb13-4381-bb48-7596d86978d4
10852024-09-06T23:36:36.091ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_dbf82130-678a-44d4-9dd9-8a8be286ccf6") failed: Property not found. retry in 673.807665ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_dbf82130-678a-44d4-9dd9-8a8be286ccf6
10862024-09-06T23:36:36.104ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2ac4364c-7933-4568-bbab-cf3f9ec0ff29") failed: Property not found. retry in 1.053434885s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2ac4364c-7933-4568-bbab-cf3f9ec0ff29
10872024-09-06T23:36:36.162ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6") failed: Property not found. retry in 1.30189266s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6
10882024-09-06T23:36:36.163ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_6c363cb4-229c-47a8-9d81-abd1714293f2: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService26'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::b'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte2'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10892024-09-06T23:36:36.163ZINFOSledAgent (ServiceManager): Booting oxz_nexus_6c363cb4-229c-47a8-9d81-abd1714293f2 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_6c363cb4-229c-47a8-9d81-abd1714293f2
10902024-09-06T23:36:36.174ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_c1d2ae9b-e921-4256-b95c-a29d6126fd66: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService23'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::1a'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_907237be-faec-4233-8c3e-1918ec810764/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::1a'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='b7c0e7fa-334a-4316-ba38-b2464ba9a394'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10912024-09-06T23:36:36.175ZINFOSledAgent (ServiceManager): Booting oxz_crucible_c1d2ae9b-e921-4256-b95c-a29d6126fd66 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_c1d2ae9b-e921-4256-b95c-a29d6126fd66
10922024-09-06T23:36:36.227ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_764a9699-9635-421f-88c6-7f14971487fe") failed: Property not found. retry in 39.386416ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_764a9699-9635-421f-88c6-7f14971487fe
10932024-09-06T23:36:36.277ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_764a9699-9635-421f-88c6-7f14971487fe") failed: Property not found. retry in 106.562847ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_764a9699-9635-421f-88c6-7f14971487fe
10942024-09-06T23:36:36.322ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_072c2fe9-5a06-4344-a93a-991a46098bd3") failed: Property not found. retry in 1.364574094s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_072c2fe9-5a06-4344-a93a-991a46098bd3
10952024-09-06T23:36:36.332ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_9ad0f243-aeb1-455d-97ad-7dc9ca9c417c") failed: Property not found. retry in 1.165146336s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_9ad0f243-aeb1-455d-97ad-7dc9ca9c417c
10962024-09-06T23:36:36.380ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de") failed: Property not found. retry in 582.225017ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de
10972024-09-06T23:36:36.389ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_6c363cb4-229c-47a8-9d81-abd1714293f2") failed: Property not found. retry in 70.448522ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_6c363cb4-229c-47a8-9d81-abd1714293f2
10982024-09-06T23:36:36.401ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_764a9699-9635-421f-88c6-7f14971487fe") failed: Property not found. retry in 112.838891ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_764a9699-9635-421f-88c6-7f14971487fe
10992024-09-06T23:36:36.412ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e273cca1-6e0d-4509-842e-40a51b28e236") failed: Property not found. retry in 1.05292949s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e273cca1-6e0d-4509-842e-40a51b28e236
11002024-09-06T23:36:36.423ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c1d2ae9b-e921-4256-b95c-a29d6126fd66") failed: Property not found. retry in 28.257034ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c1d2ae9b-e921-4256-b95c-a29d6126fd66
11012024-09-06T23:36:36.433ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0a5539c1-7e26-4e50-b26a-60a137388ce1") failed: Property not found. retry in 933.42712ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0a5539c1-7e26-4e50-b26a-60a137388ce1
11022024-09-06T23:36:36.444ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6bc5dbb5-a884-4ae2-af09-70bc587bd76b") failed: Property not found. retry in 689.484676ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6bc5dbb5-a884-4ae2-af09-70bc587bd76b
11032024-09-06T23:36:36.463ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c1d2ae9b-e921-4256-b95c-a29d6126fd66") failed: Property not found. retry in 104.620813ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c1d2ae9b-e921-4256-b95c-a29d6126fd66
11042024-09-06T23:36:36.472ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_6c363cb4-229c-47a8-9d81-abd1714293f2") failed: Property not found. retry in 69.077196ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_6c363cb4-229c-47a8-9d81-abd1714293f2
11052024-09-06T23:36:36.526ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_764a9699-9635-421f-88c6-7f14971487fe") failed: Property not found. retry in 462.89362ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_764a9699-9635-421f-88c6-7f14971487fe
11062024-09-06T23:36:36.554ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_6c363cb4-229c-47a8-9d81-abd1714293f2") failed: Property not found. retry in 242.766882ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_6c363cb4-229c-47a8-9d81-abd1714293f2
11072024-09-06T23:36:36.772ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c1d2ae9b-e921-4256-b95c-a29d6126fd66") failed: Property not found. retry in 283.333218ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c1d2ae9b-e921-4256-b95c-a29d6126fd66
11082024-09-06T23:36:36.830ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_bc144d23-6ff5-421e-a179-41beb5b46c0f") failed: Property not found. retry in 588.976122ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_bc144d23-6ff5-421e-a179-41beb5b46c0f
11092024-09-06T23:36:36.862ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_6c363cb4-229c-47a8-9d81-abd1714293f2") failed: Property not found. retry in 347.928936ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_6c363cb4-229c-47a8-9d81-abd1714293f2
11102024-09-06T23:36:36.914ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fa9373bc-3206-40d0-b421-9f54e256513c") failed: Property not found. retry in 802.1024ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fa9373bc-3206-40d0-b421-9f54e256513c
11112024-09-06T23:36:36.945ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_dbf82130-678a-44d4-9dd9-8a8be286ccf6") failed: Property not found. retry in 1.205359019s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_dbf82130-678a-44d4-9dd9-8a8be286ccf6
11122024-09-06T23:36:36.981ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ebce0088-396f-4300-91e9-19979170faa6") failed: Property not found. retry in 561.305898ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ebce0088-396f-4300-91e9-19979170faa6
11132024-09-06T23:36:36.998ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_195e4e82-1cc4-4f3b-a893-c1474bd26575") failed: Property not found. retry in 1.226454035s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_195e4e82-1cc4-4f3b-a893-c1474bd26575
11142024-09-06T23:36:37.012ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de") failed: Property not found. retry in 526.689763ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de
11152024-09-06T23:36:37.029ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_764a9699-9635-421f-88c6-7f14971487fe") failed: Property not found. retry in 823.5859ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_764a9699-9635-421f-88c6-7f14971487fe
11162024-09-06T23:36:37.069ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c1d2ae9b-e921-4256-b95c-a29d6126fd66") failed: Property not found. retry in 200.041454ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c1d2ae9b-e921-4256-b95c-a29d6126fd66
11172024-09-06T23:36:37.092ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2adfe4a1-4baf-4cb8-a3d2-e6907b84545a") failed: Property not found. retry in 963.188471ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2adfe4a1-4baf-4cb8-a3d2-e6907b84545a
11182024-09-06T23:36:37.148ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6bc5dbb5-a884-4ae2-af09-70bc587bd76b") failed: Property not found. retry in 817.917761ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6bc5dbb5-a884-4ae2-af09-70bc587bd76b
11192024-09-06T23:36:37.171ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2ac4364c-7933-4568-bbab-cf3f9ec0ff29") failed: Property not found. retry in 735.398718ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2ac4364c-7933-4568-bbab-cf3f9ec0ff29
11202024-09-06T23:36:37.222ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_6c363cb4-229c-47a8-9d81-abd1714293f2") failed: Property not found. retry in 952.456479ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_6c363cb4-229c-47a8-9d81-abd1714293f2
11212024-09-06T23:36:37.246ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_cb51aad1-b6ab-44e1-8492-82c028d74c7f: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService27'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::c'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte5'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.7'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11222024-09-06T23:36:37.246ZINFOSledAgent (ServiceManager): Booting oxz_nexus_cb51aad1-b6ab-44e1-8492-82c028d74c7f zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_cb51aad1-b6ab-44e1-8492-82c028d74c7f
11232024-09-06T23:36:37.282ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c1d2ae9b-e921-4256-b95c-a29d6126fd66") failed: Property not found. retry in 525.40825ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c1d2ae9b-e921-4256-b95c-a29d6126fd66
11242024-09-06T23:36:37.406ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cea6def4-eb13-4381-bb48-7596d86978d4") failed: Property not found. retry in 737.491803ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cea6def4-eb13-4381-bb48-7596d86978d4
11252024-09-06T23:36:37.471ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0a5539c1-7e26-4e50-b26a-60a137388ce1") failed: Property not found. retry in 1.048811964s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0a5539c1-7e26-4e50-b26a-60a137388ce1
11262024-09-06T23:36:37.483ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_bc144d23-6ff5-421e-a179-41beb5b46c0f") failed: Property not found. retry in 1.176304515s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_bc144d23-6ff5-421e-a179-41beb5b46c0f
11272024-09-06T23:36:37.497ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6") failed: Property not found. retry in 1.122094611s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6
11282024-09-06T23:36:37.511ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e273cca1-6e0d-4509-842e-40a51b28e236") failed: Property not found. retry in 1.16432992s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e273cca1-6e0d-4509-842e-40a51b28e236
11292024-09-06T23:36:37.524ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_9ad0f243-aeb1-455d-97ad-7dc9ca9c417c") failed: Property not found. retry in 926.216118ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_9ad0f243-aeb1-455d-97ad-7dc9ca9c417c
11302024-09-06T23:36:37.551ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de") failed: Property not found. retry in 721.710667ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de
11312024-09-06T23:36:37.562ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ebce0088-396f-4300-91e9-19979170faa6") failed: Property not found. retry in 888.326479ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ebce0088-396f-4300-91e9-19979170faa6
11322024-09-06T23:36:37.573ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_cb51aad1-b6ab-44e1-8492-82c028d74c7f") failed: Property not found. retry in 33.391475ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_cb51aad1-b6ab-44e1-8492-82c028d74c7f
11332024-09-06T23:36:37.619ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_cb51aad1-b6ab-44e1-8492-82c028d74c7f") failed: Property not found. retry in 131.191249ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_cb51aad1-b6ab-44e1-8492-82c028d74c7f
11342024-09-06T23:36:37.701ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_072c2fe9-5a06-4344-a93a-991a46098bd3") failed: Property not found. retry in 1.409036008s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_072c2fe9-5a06-4344-a93a-991a46098bd3
11352024-09-06T23:36:37.729ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fa9373bc-3206-40d0-b421-9f54e256513c") failed: Property not found. retry in 889.999356ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fa9373bc-3206-40d0-b421-9f54e256513c
11362024-09-06T23:36:37.761ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_cb51aad1-b6ab-44e1-8492-82c028d74c7f") failed: Property not found. retry in 222.772987ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_cb51aad1-b6ab-44e1-8492-82c028d74c7f
11372024-09-06T23:36:37.822ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c1d2ae9b-e921-4256-b95c-a29d6126fd66") failed: Property not found. retry in 1.306128742s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c1d2ae9b-e921-4256-b95c-a29d6126fd66
11382024-09-06T23:36:37.868ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_764a9699-9635-421f-88c6-7f14971487fe") failed: Property not found. retry in 1.054566175s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_764a9699-9635-421f-88c6-7f14971487fe
11392024-09-06T23:36:37.920ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2ac4364c-7933-4568-bbab-cf3f9ec0ff29") failed: Property not found. retry in 1.431124882s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2ac4364c-7933-4568-bbab-cf3f9ec0ff29
11402024-09-06T23:36:37.979ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6bc5dbb5-a884-4ae2-af09-70bc587bd76b") failed: Property not found. retry in 1.479043451s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6bc5dbb5-a884-4ae2-af09-70bc587bd76b
11412024-09-06T23:36:37.996ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_cb51aad1-b6ab-44e1-8492-82c028d74c7f") failed: Property not found. retry in 496.177305ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_cb51aad1-b6ab-44e1-8492-82c028d74c7f
11422024-09-06T23:36:38.065ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2adfe4a1-4baf-4cb8-a3d2-e6907b84545a") failed: Property not found. retry in 1.253404695s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2adfe4a1-4baf-4cb8-a3d2-e6907b84545a
11432024-09-06T23:36:38.154ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cea6def4-eb13-4381-bb48-7596d86978d4") failed: Property not found. retry in 1.489227302s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cea6def4-eb13-4381-bb48-7596d86978d4
11442024-09-06T23:36:38.162ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_dbf82130-678a-44d4-9dd9-8a8be286ccf6") failed: Property not found. retry in 542.78839ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_dbf82130-678a-44d4-9dd9-8a8be286ccf6
11452024-09-06T23:36:38.185ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_6c363cb4-229c-47a8-9d81-abd1714293f2") failed: Property not found. retry in 974.517491ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_6c363cb4-229c-47a8-9d81-abd1714293f2
11462024-09-06T23:36:38.234ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_195e4e82-1cc4-4f3b-a893-c1474bd26575") failed: Property not found. retry in 1.064231586s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_195e4e82-1cc4-4f3b-a893-c1474bd26575
11472024-09-06T23:36:38.282ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de") failed: Property not found. retry in 826.990601ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de
11482024-09-06T23:36:38.460ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_9ad0f243-aeb1-455d-97ad-7dc9ca9c417c") failed: Property not found. retry in 553.49214ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_9ad0f243-aeb1-455d-97ad-7dc9ca9c417c
11492024-09-06T23:36:38.471ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ebce0088-396f-4300-91e9-19979170faa6") failed: Property not found. retry in 1.361202124s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ebce0088-396f-4300-91e9-19979170faa6
11502024-09-06T23:36:38.503ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_cb51aad1-b6ab-44e1-8492-82c028d74c7f") failed: Property not found. retry in 902.943761ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_cb51aad1-b6ab-44e1-8492-82c028d74c7f
11512024-09-06T23:36:38.530ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0a5539c1-7e26-4e50-b26a-60a137388ce1") failed: Property not found. retry in 1.238033553s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0a5539c1-7e26-4e50-b26a-60a137388ce1
11522024-09-06T23:36:38.632ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6") failed: Property not found. retry in 952.660495ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6
11532024-09-06T23:36:38.642ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fa9373bc-3206-40d0-b421-9f54e256513c") failed: Property not found. retry in 1.479732332s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fa9373bc-3206-40d0-b421-9f54e256513c
11542024-09-06T23:36:38.672ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_bc144d23-6ff5-421e-a179-41beb5b46c0f") failed: Property not found. retry in 623.935729ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_bc144d23-6ff5-421e-a179-41beb5b46c0f
11552024-09-06T23:36:38.690ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e273cca1-6e0d-4509-842e-40a51b28e236") failed: Property not found. retry in 647.799954ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e273cca1-6e0d-4509-842e-40a51b28e236
11562024-09-06T23:36:38.717ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_dbf82130-678a-44d4-9dd9-8a8be286ccf6") failed: Property not found. retry in 1.090683545s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_dbf82130-678a-44d4-9dd9-8a8be286ccf6
11572024-09-06T23:36:38.937ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_764a9699-9635-421f-88c6-7f14971487fe") failed: Property not found. retry in 565.935533ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_764a9699-9635-421f-88c6-7f14971487fe
11582024-09-06T23:36:39.027ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_9ad0f243-aeb1-455d-97ad-7dc9ca9c417c") failed: Property not found. retry in 1.068936512s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_9ad0f243-aeb1-455d-97ad-7dc9ca9c417c
11592024-09-06T23:36:39.121ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de") failed: Property not found. retry in 613.202914ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de
11602024-09-06T23:36:39.135ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_072c2fe9-5a06-4344-a93a-991a46098bd3") failed: Property not found. retry in 1.384003986s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_072c2fe9-5a06-4344-a93a-991a46098bd3
11612024-09-06T23:36:39.149ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c1d2ae9b-e921-4256-b95c-a29d6126fd66") failed: Property not found. retry in 524.931227ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c1d2ae9b-e921-4256-b95c-a29d6126fd66
11622024-09-06T23:36:39.173ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_6c363cb4-229c-47a8-9d81-abd1714293f2") failed: Property not found. retry in 798.757118ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_6c363cb4-229c-47a8-9d81-abd1714293f2
11632024-09-06T23:36:39.308ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_bc144d23-6ff5-421e-a179-41beb5b46c0f") failed: Property not found. retry in 510.948526ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_bc144d23-6ff5-421e-a179-41beb5b46c0f
11642024-09-06T23:36:39.316ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_195e4e82-1cc4-4f3b-a893-c1474bd26575") failed: Property not found. retry in 698.436145ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_195e4e82-1cc4-4f3b-a893-c1474bd26575
11652024-09-06T23:36:39.329ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2adfe4a1-4baf-4cb8-a3d2-e6907b84545a") failed: Property not found. retry in 1.169822028s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2adfe4a1-4baf-4cb8-a3d2-e6907b84545a
11662024-09-06T23:36:39.350ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e273cca1-6e0d-4509-842e-40a51b28e236") failed: Property not found. retry in 1.424813389s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e273cca1-6e0d-4509-842e-40a51b28e236
11672024-09-06T23:36:39.361ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2ac4364c-7933-4568-bbab-cf3f9ec0ff29") failed: Property not found. retry in 671.983331ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2ac4364c-7933-4568-bbab-cf3f9ec0ff29
11682024-09-06T23:36:39.418ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_cb51aad1-b6ab-44e1-8492-82c028d74c7f") failed: Property not found. retry in 793.78986ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_cb51aad1-b6ab-44e1-8492-82c028d74c7f
11692024-09-06T23:36:39.470ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6bc5dbb5-a884-4ae2-af09-70bc587bd76b") failed: Property not found. retry in 1.376068598s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6bc5dbb5-a884-4ae2-af09-70bc587bd76b
11702024-09-06T23:36:39.515ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_764a9699-9635-421f-88c6-7f14971487fe") failed: Property not found. retry in 926.585621ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_764a9699-9635-421f-88c6-7f14971487fe
11712024-09-06T23:36:39.597ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6") failed: Property not found. retry in 548.737286ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6
11722024-09-06T23:36:39.657ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cea6def4-eb13-4381-bb48-7596d86978d4") failed: Property not found. retry in 1.136742122s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cea6def4-eb13-4381-bb48-7596d86978d4
11732024-09-06T23:36:39.687ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c1d2ae9b-e921-4256-b95c-a29d6126fd66") failed: Property not found. retry in 1.349189917s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c1d2ae9b-e921-4256-b95c-a29d6126fd66
11742024-09-06T23:36:39.746ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de") failed: Property not found. retry in 1.184426055s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de
11752024-09-06T23:36:39.779ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0a5539c1-7e26-4e50-b26a-60a137388ce1") failed: Property not found. retry in 792.683332ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0a5539c1-7e26-4e50-b26a-60a137388ce1
11762024-09-06T23:36:39.818ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_dbf82130-678a-44d4-9dd9-8a8be286ccf6") failed: Property not found. retry in 967.098779ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_dbf82130-678a-44d4-9dd9-8a8be286ccf6
11772024-09-06T23:36:39.830ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_bc144d23-6ff5-421e-a179-41beb5b46c0f") failed: Property not found. retry in 1.29451293s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_bc144d23-6ff5-421e-a179-41beb5b46c0f
11782024-09-06T23:36:39.842ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ebce0088-396f-4300-91e9-19979170faa6") failed: Property not found. retry in 911.779957ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ebce0088-396f-4300-91e9-19979170faa6
11792024-09-06T23:36:39.981ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_6c363cb4-229c-47a8-9d81-abd1714293f2") failed: Property not found. retry in 1.382800434s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_6c363cb4-229c-47a8-9d81-abd1714293f2
11802024-09-06T23:36:40.027ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_195e4e82-1cc4-4f3b-a893-c1474bd26575") failed: Property not found. retry in 748.932012ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_195e4e82-1cc4-4f3b-a893-c1474bd26575
11812024-09-06T23:36:40.044ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2ac4364c-7933-4568-bbab-cf3f9ec0ff29") failed: Property not found. retry in 1.447567598s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2ac4364c-7933-4568-bbab-cf3f9ec0ff29
11822024-09-06T23:36:40.106ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_9ad0f243-aeb1-455d-97ad-7dc9ca9c417c") failed: Property not found. retry in 507.295137ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_9ad0f243-aeb1-455d-97ad-7dc9ca9c417c
11832024-09-06T23:36:40.130ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fa9373bc-3206-40d0-b421-9f54e256513c") failed: Property not found. retry in 677.539771ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fa9373bc-3206-40d0-b421-9f54e256513c
11842024-09-06T23:36:40.156ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6") failed: Property not found. retry in 1.064569237s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6
11852024-09-06T23:36:40.222ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_cb51aad1-b6ab-44e1-8492-82c028d74c7f") failed: Property not found. retry in 1.410606316s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_cb51aad1-b6ab-44e1-8492-82c028d74c7f
11862024-09-06T23:36:40.453ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_764a9699-9635-421f-88c6-7f14971487fe") failed: Property not found. retry in 1.031997808s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_764a9699-9635-421f-88c6-7f14971487fe
11872024-09-06T23:36:40.509ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2adfe4a1-4baf-4cb8-a3d2-e6907b84545a") failed: Property not found. retry in 836.631578ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2adfe4a1-4baf-4cb8-a3d2-e6907b84545a
11882024-09-06T23:36:40.528ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_072c2fe9-5a06-4344-a93a-991a46098bd3") failed: Property not found. retry in 517.085096ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_072c2fe9-5a06-4344-a93a-991a46098bd3
11892024-09-06T23:36:40.580ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0a5539c1-7e26-4e50-b26a-60a137388ce1") failed: Property not found. retry in 622.052762ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0a5539c1-7e26-4e50-b26a-60a137388ce1
11902024-09-06T23:36:40.624ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_9ad0f243-aeb1-455d-97ad-7dc9ca9c417c") failed: Property not found. retry in 1.039233273s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_9ad0f243-aeb1-455d-97ad-7dc9ca9c417c
11912024-09-06T23:36:40.763ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ebce0088-396f-4300-91e9-19979170faa6") failed: Property not found. retry in 791.319808ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ebce0088-396f-4300-91e9-19979170faa6
11922024-09-06T23:36:40.785ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e273cca1-6e0d-4509-842e-40a51b28e236") failed: Property not found. retry in 578.323424ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e273cca1-6e0d-4509-842e-40a51b28e236
11932024-09-06T23:36:40.785ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 1.45186146s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c
11942024-09-06T23:36:40.794ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_195e4e82-1cc4-4f3b-a893-c1474bd26575") failed: Property not found. retry in 819.789614ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_195e4e82-1cc4-4f3b-a893-c1474bd26575
11952024-09-06T23:36:40.802ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_dbf82130-678a-44d4-9dd9-8a8be286ccf6") failed: Property not found. retry in 1.266530353s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_dbf82130-678a-44d4-9dd9-8a8be286ccf6
11962024-09-06T23:36:40.810ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cea6def4-eb13-4381-bb48-7596d86978d4") failed: Property not found. retry in 621.860252ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cea6def4-eb13-4381-bb48-7596d86978d4
11972024-09-06T23:36:40.819ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fa9373bc-3206-40d0-b421-9f54e256513c") failed: Property not found. retry in 1.07189956s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fa9373bc-3206-40d0-b421-9f54e256513c
11982024-09-06T23:36:40.857ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6bc5dbb5-a884-4ae2-af09-70bc587bd76b") failed: Property not found. retry in 735.870816ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6bc5dbb5-a884-4ae2-af09-70bc587bd76b
11992024-09-06T23:36:40.940ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de") failed: Property not found. retry in 949.976221ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de
12002024-09-06T23:36:41.048ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c1d2ae9b-e921-4256-b95c-a29d6126fd66") failed: Property not found. retry in 669.131027ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c1d2ae9b-e921-4256-b95c-a29d6126fd66
12012024-09-06T23:36:41.058ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_072c2fe9-5a06-4344-a93a-991a46098bd3") failed: Property not found. retry in 1.470313286s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_072c2fe9-5a06-4344-a93a-991a46098bd3
12022024-09-06T23:36:41.138ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_bc144d23-6ff5-421e-a179-41beb5b46c0f") failed: Property not found. retry in 1.414338069s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_bc144d23-6ff5-421e-a179-41beb5b46c0f
12032024-09-06T23:36:41.213ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0a5539c1-7e26-4e50-b26a-60a137388ce1") failed: Property not found. retry in 1.212791856s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0a5539c1-7e26-4e50-b26a-60a137388ce1
12042024-09-06T23:36:41.230ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6") failed: Property not found. retry in 847.841624ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6
12052024-09-06T23:36:41.361ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2adfe4a1-4baf-4cb8-a3d2-e6907b84545a") failed: Property not found. retry in 1.361991188s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2adfe4a1-4baf-4cb8-a3d2-e6907b84545a
12062024-09-06T23:36:41.377ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_6c363cb4-229c-47a8-9d81-abd1714293f2") failed: Property not found. retry in 1.294469259s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_6c363cb4-229c-47a8-9d81-abd1714293f2
12072024-09-06T23:36:41.388ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e273cca1-6e0d-4509-842e-40a51b28e236") failed: Property not found. retry in 1.155117309s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e273cca1-6e0d-4509-842e-40a51b28e236
12082024-09-06T23:36:41.444ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cea6def4-eb13-4381-bb48-7596d86978d4") failed: Property not found. retry in 1.168688038s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cea6def4-eb13-4381-bb48-7596d86978d4
12092024-09-06T23:36:41.496ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_764a9699-9635-421f-88c6-7f14971487fe") failed: Property not found. retry in 842.260125ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_764a9699-9635-421f-88c6-7f14971487fe
12102024-09-06T23:36:41.506ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2ac4364c-7933-4568-bbab-cf3f9ec0ff29") failed: Property not found. retry in 512.2839ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2ac4364c-7933-4568-bbab-cf3f9ec0ff29
12112024-09-06T23:36:41.567ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ebce0088-396f-4300-91e9-19979170faa6") failed: Property not found. retry in 1.085266537s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ebce0088-396f-4300-91e9-19979170faa6
12122024-09-06T23:36:41.608ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6bc5dbb5-a884-4ae2-af09-70bc587bd76b") failed: Property not found. retry in 932.697992ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6bc5dbb5-a884-4ae2-af09-70bc587bd76b
12132024-09-06T23:36:41.626ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_195e4e82-1cc4-4f3b-a893-c1474bd26575") failed: Property not found. retry in 1.475660272s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_195e4e82-1cc4-4f3b-a893-c1474bd26575
12142024-09-06T23:36:41.646ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_cb51aad1-b6ab-44e1-8492-82c028d74c7f") failed: Property not found. retry in 1.301415206s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_cb51aad1-b6ab-44e1-8492-82c028d74c7f
12152024-09-06T23:36:41.676ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_9ad0f243-aeb1-455d-97ad-7dc9ca9c417c") failed: Property not found. retry in 665.808665ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_9ad0f243-aeb1-455d-97ad-7dc9ca9c417c
12162024-09-06T23:36:41.731ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c1d2ae9b-e921-4256-b95c-a29d6126fd66") failed: Property not found. retry in 1.00994531s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c1d2ae9b-e921-4256-b95c-a29d6126fd66
12172024-09-06T23:36:41.904ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de") failed: Property not found. retry in 1.298398611s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de
12182024-09-06T23:36:41.919ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fa9373bc-3206-40d0-b421-9f54e256513c") failed: Property not found. retry in 1.22290581s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fa9373bc-3206-40d0-b421-9f54e256513c
12192024-09-06T23:36:42.032ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2ac4364c-7933-4568-bbab-cf3f9ec0ff29") failed: Property not found. retry in 1.379126833s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2ac4364c-7933-4568-bbab-cf3f9ec0ff29
12202024-09-06T23:36:42.086ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_dbf82130-678a-44d4-9dd9-8a8be286ccf6") failed: Property not found. retry in 1.24857082s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_dbf82130-678a-44d4-9dd9-8a8be286ccf6
12212024-09-06T23:36:42.100ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6") failed: Property not found. retry in 585.490587ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6
12222024-09-06T23:36:42.358ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_764a9699-9635-421f-88c6-7f14971487fe") failed: Property not found. retry in 540.98368ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_764a9699-9635-421f-88c6-7f14971487fe
12232024-09-06T23:36:42.374ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_9ad0f243-aeb1-455d-97ad-7dc9ca9c417c") failed: Property not found. retry in 751.454988ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_9ad0f243-aeb1-455d-97ad-7dc9ca9c417c
12242024-09-06T23:36:42.562ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6bc5dbb5-a884-4ae2-af09-70bc587bd76b") failed: Property not found. retry in 1.312906496s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6bc5dbb5-a884-4ae2-af09-70bc587bd76b
12252024-09-06T23:36:42.581ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e273cca1-6e0d-4509-842e-40a51b28e236") failed: Property not found. retry in 1.14786329s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e273cca1-6e0d-4509-842e-40a51b28e236
12262024-09-06T23:36:42.709ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_6c363cb4-229c-47a8-9d81-abd1714293f2") failed: Property not found. retry in 1.083377747s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_6c363cb4-229c-47a8-9d81-abd1714293f2
12272024-09-06T23:36:42.722ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6") failed: Property not found. retry in 1.300744072s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6
12282024-09-06T23:36:42.756ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c1d2ae9b-e921-4256-b95c-a29d6126fd66") failed: Property not found. retry in 1.302281364s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c1d2ae9b-e921-4256-b95c-a29d6126fd66
12292024-09-06T23:36:42.916ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_764a9699-9635-421f-88c6-7f14971487fe") failed: Property not found. retry in 859.618572ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_764a9699-9635-421f-88c6-7f14971487fe
12302024-09-06T23:36:42.964ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_cb51aad1-b6ab-44e1-8492-82c028d74c7f") failed: Property not found. retry in 1.392064066s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_cb51aad1-b6ab-44e1-8492-82c028d74c7f
12312024-09-06T23:36:43.122ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_195e4e82-1cc4-4f3b-a893-c1474bd26575") failed: Property not found. retry in 1.144065107s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_195e4e82-1cc4-4f3b-a893-c1474bd26575
12322024-09-06T23:36:43.142ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_9ad0f243-aeb1-455d-97ad-7dc9ca9c417c") failed: Property not found. retry in 602.155716ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_9ad0f243-aeb1-455d-97ad-7dc9ca9c417c
12332024-09-06T23:36:43.218ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de") failed: Property not found. retry in 914.783042ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de
12342024-09-06T23:36:43.431ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2ac4364c-7933-4568-bbab-cf3f9ec0ff29") failed: Property not found. retry in 1.274374363s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2ac4364c-7933-4568-bbab-cf3f9ec0ff29
12352024-09-06T23:36:43.745ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e273cca1-6e0d-4509-842e-40a51b28e236") failed: Property not found. retry in 776.763921ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e273cca1-6e0d-4509-842e-40a51b28e236
12362024-09-06T23:36:43.762ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_9ad0f243-aeb1-455d-97ad-7dc9ca9c417c") failed: Property not found. retry in 1.327438714s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_9ad0f243-aeb1-455d-97ad-7dc9ca9c417c
12372024-09-06T23:36:43.791ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_764a9699-9635-421f-88c6-7f14971487fe") failed: Property not found. retry in 1.032999474s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_764a9699-9635-421f-88c6-7f14971487fe
12382024-09-06T23:36:43.812ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_6c363cb4-229c-47a8-9d81-abd1714293f2") failed: Property not found. retry in 599.346036ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_6c363cb4-229c-47a8-9d81-abd1714293f2
12392024-09-06T23:36:44.040ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6") failed: Property not found. retry in 545.695209ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6
12402024-09-06T23:36:44.072ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c1d2ae9b-e921-4256-b95c-a29d6126fd66") failed: Property not found. retry in 1.206064837s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c1d2ae9b-e921-4256-b95c-a29d6126fd66
12412024-09-06T23:36:44.150ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de") failed: Property not found. retry in 733.356048ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de
12422024-09-06T23:36:44.282ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_195e4e82-1cc4-4f3b-a893-c1474bd26575") failed: Property not found. retry in 1.144662151s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_195e4e82-1cc4-4f3b-a893-c1474bd26575
12432024-09-06T23:36:44.372ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_cb51aad1-b6ab-44e1-8492-82c028d74c7f") failed: Property not found. retry in 1.257152013s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_cb51aad1-b6ab-44e1-8492-82c028d74c7f
12442024-09-06T23:36:44.428ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_6c363cb4-229c-47a8-9d81-abd1714293f2") failed: Property not found. retry in 1.478708161s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_6c363cb4-229c-47a8-9d81-abd1714293f2
12452024-09-06T23:36:44.542ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e273cca1-6e0d-4509-842e-40a51b28e236") failed: Property not found. retry in 1.446207734s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e273cca1-6e0d-4509-842e-40a51b28e236
12462024-09-06T23:36:44.604ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6") failed: Property not found. retry in 678.35844ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6
12472024-09-06T23:36:44.718ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2ac4364c-7933-4568-bbab-cf3f9ec0ff29") failed: Property not found. retry in 1.128726503s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2ac4364c-7933-4568-bbab-cf3f9ec0ff29
12482024-09-06T23:36:44.837ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_764a9699-9635-421f-88c6-7f14971487fe") failed: Property not found. retry in 1.293822116s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_764a9699-9635-421f-88c6-7f14971487fe
12492024-09-06T23:36:44.896ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de") failed: Property not found. retry in 1.040579835s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de
12502024-09-06T23:36:45.103ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_9ad0f243-aeb1-455d-97ad-7dc9ca9c417c") failed: Property not found. retry in 931.394424ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_9ad0f243-aeb1-455d-97ad-7dc9ca9c417c
12512024-09-06T23:36:45.292ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c1d2ae9b-e921-4256-b95c-a29d6126fd66") failed: Property not found. retry in 1.464116938s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c1d2ae9b-e921-4256-b95c-a29d6126fd66
12522024-09-06T23:36:45.307ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6") failed: Property not found. retry in 1.14280822s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6
12532024-09-06T23:36:45.442ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_195e4e82-1cc4-4f3b-a893-c1474bd26575") failed: Property not found. retry in 508.678817ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_195e4e82-1cc4-4f3b-a893-c1474bd26575
12542024-09-06T23:36:45.643ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_cb51aad1-b6ab-44e1-8492-82c028d74c7f") failed: Property not found. retry in 1.087927584s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_cb51aad1-b6ab-44e1-8492-82c028d74c7f
12552024-09-06T23:36:45.859ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2ac4364c-7933-4568-bbab-cf3f9ec0ff29") failed: Property not found. retry in 1.465301147s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2ac4364c-7933-4568-bbab-cf3f9ec0ff29
12562024-09-06T23:36:45.921ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_6c363cb4-229c-47a8-9d81-abd1714293f2") failed: Property not found. retry in 1.084843256s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_6c363cb4-229c-47a8-9d81-abd1714293f2
12572024-09-06T23:36:45.966ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_195e4e82-1cc4-4f3b-a893-c1474bd26575") failed: Property not found. retry in 1.397124789s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_195e4e82-1cc4-4f3b-a893-c1474bd26575
12582024-09-06T23:36:46.000ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e273cca1-6e0d-4509-842e-40a51b28e236") failed: Property not found. retry in 1.363633094s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e273cca1-6e0d-4509-842e-40a51b28e236
12592024-09-06T23:36:46.047ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_9ad0f243-aeb1-455d-97ad-7dc9ca9c417c") failed: Property not found. retry in 770.739756ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_9ad0f243-aeb1-455d-97ad-7dc9ca9c417c
12602024-09-06T23:36:46.142ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_764a9699-9635-421f-88c6-7f14971487fe") failed: Property not found. retry in 714.526763ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_764a9699-9635-421f-88c6-7f14971487fe
12612024-09-06T23:36:46.460ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6") failed: Property not found. retry in 1.344565494s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6
12622024-09-06T23:36:46.742ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_cb51aad1-b6ab-44e1-8492-82c028d74c7f") failed: Property not found. retry in 1.383179129s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_cb51aad1-b6ab-44e1-8492-82c028d74c7f
12632024-09-06T23:36:46.767ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c1d2ae9b-e921-4256-b95c-a29d6126fd66") failed: Property not found. retry in 1.245883579s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c1d2ae9b-e921-4256-b95c-a29d6126fd66
12642024-09-06T23:36:47.019ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_6c363cb4-229c-47a8-9d81-abd1714293f2") failed: Property not found. retry in 1.053728283s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_6c363cb4-229c-47a8-9d81-abd1714293f2
12652024-09-06T23:36:47.605ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 1.697408004s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers): error trying to connect: tcp connect error: Connection refused (os error 146)" file = oximeter/producer/src/lib.rs:424 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c
12662024-09-06T23:36:48.165ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3226 zone = oxz_oximeter_bc144d23-6ff5-421e-a179-41beb5b46c0f
12672024-09-06T23:36:48.166ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3226 zone = oxz_nexus_cb51aad1-b6ab-44e1-8492-82c028d74c7f
12682024-09-06T23:36:48.166ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3226 zone = oxz_crucible_pantry_b047cd23-d622-477e-885d-d1b3824512de
12692024-09-06T23:36:48.166ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3226 zone = oxz_crucible_pantry_072c2fe9-5a06-4344-a93a-991a46098bd3
12702024-09-06T23:36:48.166ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3226 zone = oxz_crucible_cea6def4-eb13-4381-bb48-7596d86978d4
12712024-09-06T23:36:48.166ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3226 zone = oxz_nexus_9ad0f243-aeb1-455d-97ad-7dc9ca9c417c
12722024-09-06T23:36:48.166ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3226 zone = oxz_crucible_0a5539c1-7e26-4e50-b26a-60a137388ce1
12732024-09-06T23:36:48.166ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3226 zone = oxz_nexus_6c363cb4-229c-47a8-9d81-abd1714293f2
12742024-09-06T23:36:48.166ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3226 zone = oxz_external_dns_5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6
12752024-09-06T23:36:48.166ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3226 zone = oxz_clickhouse_2ac4364c-7933-4568-bbab-cf3f9ec0ff29
12762024-09-06T23:36:48.166ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3226 zone = oxz_crucible_fa9373bc-3206-40d0-b421-9f54e256513c
12772024-09-06T23:36:48.166ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3226 zone = oxz_crucible_6bc5dbb5-a884-4ae2-af09-70bc587bd76b
12782024-09-06T23:36:48.166ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3226 zone = oxz_crucible_pantry_764a9699-9635-421f-88c6-7f14971487fe
12792024-09-06T23:36:48.166ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3226 zone = oxz_crucible_2adfe4a1-4baf-4cb8-a3d2-e6907b84545a
12802024-09-06T23:36:48.166ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3226 zone = oxz_crucible_dbf82130-678a-44d4-9dd9-8a8be286ccf6
12812024-09-06T23:36:48.166ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3226 zone = oxz_crucible_e273cca1-6e0d-4509-842e-40a51b28e236
12822024-09-06T23:36:48.166ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3226 zone = oxz_crucible_ebce0088-396f-4300-91e9-19979170faa6
12832024-09-06T23:36:48.166ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3226 zone = oxz_external_dns_195e4e82-1cc4-4f3b-a893-c1474bd26575
12842024-09-06T23:36:48.166ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3226 zone = oxz_crucible_c1d2ae9b-e921-4256-b95c-a29d6126fd66
12852024-09-06T23:36:48.166ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/ca24db04-f37b-40b1-83c2-23476bec5a4a/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
12862024-09-06T23:36:48.167ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/358fc814-3f2a-4b1a-aebb-894434d72d01/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
12872024-09-06T23:36:48.169ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 23095826 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:61861 req_id = a3346c80-31b1-4ff6-8bbe-55fa75c25714 response_code = 204 uri = /omicron-zones
12882024-09-06T23:36:48.173ZINFOSledAgent (RSS): Finished setting up services file = sled-agent/src/rack_setup/service.rs:1322
12892024-09-06T23:36:48.173ZINFOSledAgent (RSS): Writing ledger to /pool/int/ca24db04-f37b-40b1-83c2-23476bec5a4a/config/.rss-plan-completed.marker.tmp file = common/src/ledger.rs:199
12902024-09-06T23:36:48.173ZINFOSledAgent (RSS): Writing ledger to /pool/int/358fc814-3f2a-4b1a-aebb-894434d72d01/config/.rss-plan-completed.marker.tmp file = common/src/ledger.rs:199
12912024-09-06T23:36:48.176ZINFOSledAgent (RSS): Handing off control to Nexus file = sled-agent/src/rack_setup/service.rs:707
12922024-09-06T23:36:48.176ZINFOSledAgent (RSS): Nexus address: [fd00:1122:3344:101::b]:12221 file = sled-agent/src/rack_setup/service.rs:720
12932024-09-06T23:36:48.222ZINFOSledAgent (RSS): rack_network_config: RackNetworkConfigV2 { bfd: [], bgp: [], infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [ PortConfigV2 { addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.2.174, width: 24, }, ), vlan_id: None, }, ], autoneg: false, bgp_peers: [], lldp: None, port: "qsfp0", routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), local_pref: None, nexthop: 10.151.2.100, vlan_id: None, }, ], switch: Switch0, uplink_port_fec: None, uplink_port_speed: Speed40G, }, ], rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, } file = sled-agent/src/rack_setup/service.rs:887
12942024-09-06T23:36:48.222ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/f8e86eb4-0496-4cd3-9bda-2c06ff0605a7/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:949
12952024-09-06T23:36:48.326ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/f8e86eb4-0496-4cd3-9bda-2c06ff0605a7/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:949
12962024-09-06T23:36:48.497ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/f8e86eb4-0496-4cd3-9bda-2c06ff0605a7/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:949
12972024-09-06T23:36:48.686ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/f8e86eb4-0496-4cd3-9bda-2c06ff0605a7/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:949
12982024-09-06T23:36:48.920ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/f8e86eb4-0496-4cd3-9bda-2c06ff0605a7/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:949
12992024-09-06T23:36:49.159ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/f8e86eb4-0496-4cd3-9bda-2c06ff0605a7/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:949
13002024-09-06T23:36:49.341ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 3.676885942s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers): error trying to connect: tcp connect error: Connection refused (os error 146)" file = oximeter/producer/src/lib.rs:424 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c
13012024-09-06T23:36:49.363ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/f8e86eb4-0496-4cd3-9bda-2c06ff0605a7/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:949
13022024-09-06T23:36:49.660ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/f8e86eb4-0496-4cd3-9bda-2c06ff0605a7/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:949
13032024-09-06T23:36:50.131ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/f8e86eb4-0496-4cd3-9bda-2c06ff0605a7/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:949
13042024-09-06T23:36:50.375ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/f8e86eb4-0496-4cd3-9bda-2c06ff0605a7/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:949
13052024-09-06T23:36:50.739ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/f8e86eb4-0496-4cd3-9bda-2c06ff0605a7/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:949
13062024-09-06T23:36:51.200ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/f8e86eb4-0496-4cd3-9bda-2c06ff0605a7/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:949
13072024-09-06T23:36:51.864ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/f8e86eb4-0496-4cd3-9bda-2c06ff0605a7/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:949
13082024-09-06T23:36:52.722ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/f8e86eb4-0496-4cd3-9bda-2c06ff0605a7/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:949
13092024-09-06T23:36:53.051ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 10.691750527s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers): error trying to connect: tcp connect error: Connection refused (os error 146)" file = oximeter/producer/src/lib.rs:424 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c
13102024-09-06T23:36:54.018ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/f8e86eb4-0496-4cd3-9bda-2c06ff0605a7/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:949
13112024-09-06T23:36:55.076ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:60502
13122024-09-06T23:36:55.076ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 39 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:60502 req_id = b1bfbe25-7ccd-480a-a2a4-6fbfb26c4a00 response_code = 200 uri = /baseboard
13132024-09-06T23:36:55.350ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/f8e86eb4-0496-4cd3-9bda-2c06ff0605a7/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:949
13142024-09-06T23:36:56.379ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/f8e86eb4-0496-4cd3-9bda-2c06ff0605a7/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:949
13152024-09-06T23:36:58.717ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/f8e86eb4-0496-4cd3-9bda-2c06ff0605a7/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:949
13162024-09-06T23:37:01.561ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/f8e86eb4-0496-4cd3-9bda-2c06ff0605a7/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:949
13172024-09-06T23:37:03.776ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 21.311644024s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers): error trying to connect: tcp connect error: Connection refused (os error 146)" file = oximeter/producer/src/lib.rs:424 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c
13182024-09-06T23:37:05.317ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/f8e86eb4-0496-4cd3-9bda-2c06ff0605a7/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:949
13192024-09-06T23:37:09.302ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "d0540884-9683-45df-aad7-6a7e83846052", "content-length": "173", "date": "Fri, 06 Sep 2024 23:37:08 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: rack with id \"f8e86eb4-0496-4cd3-9bda-2c06ff0605a7\"", request_id: "d0540884-9683-45df-aad7-6a7e83846052" } file = sled-agent/src/rack_setup/service.rs:949
13202024-09-06T23:37:12.051ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "dcad3579-0690-488c-a4d3-aca16cba5431", "content-length": "173", "date": "Fri, 06 Sep 2024 23:37:11 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: rack with id \"f8e86eb4-0496-4cd3-9bda-2c06ff0605a7\"", request_id: "dcad3579-0690-488c-a4d3-aca16cba5431" } file = sled-agent/src/rack_setup/service.rs:949
13212024-09-06T23:37:17.180ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "97c72d2d-3508-4b0d-a8da-951f7a373430", "content-length": "173", "date": "Fri, 06 Sep 2024 23:37:16 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"55267d89-b780-4916-86a8-64533b7bfd1c\"", request_id: "97c72d2d-3508-4b0d-a8da-951f7a373430" } file = sled-agent/src/rack_setup/service.rs:949
13222024-09-06T23:37:21.747ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "09f4656e-ae24-456d-a6a5-573b2de3ad0c", "content-length": "173", "date": "Fri, 06 Sep 2024 23:37:21 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"55267d89-b780-4916-86a8-64533b7bfd1c\"", request_id: "09f4656e-ae24-456d-a6a5-573b2de3ad0c" } file = sled-agent/src/rack_setup/service.rs:949
13232024-09-06T23:37:25.123ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:44727
13242024-09-06T23:37:25.123ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 37 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:44727 req_id = 5405e689-c9dc-4213-a0cd-f27699539cc8 response_code = 200 uri = /baseboard
13252024-09-06T23:37:25.803ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "354b4a4a-b316-4bc7-999a-ffbaee66833c", "content-length": "173", "date": "Fri, 06 Sep 2024 23:37:25 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"55267d89-b780-4916-86a8-64533b7bfd1c\"", request_id: "354b4a4a-b316-4bc7-999a-ffbaee66833c" } file = sled-agent/src/rack_setup/service.rs:949
13262024-09-06T23:37:37.118ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "fd79adcb-71b6-41d0-a524-7deb38d1f57c", "content-length": "173", "date": "Fri, 06 Sep 2024 23:37:36 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"55267d89-b780-4916-86a8-64533b7bfd1c\"", request_id: "fd79adcb-71b6-41d0-a524-7deb38d1f57c" } file = sled-agent/src/rack_setup/service.rs:949
13272024-09-06T23:37:41.083ZINFOSledAgent (NexusNotifierTask): Successfully put SledAgentInfo to nexus file = sled-agent/src/nexus.rs:544 new_generation = 1 old_generation = 1 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c
13282024-09-06T23:37:41.200ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:61500
13292024-09-06T23:37:41.200ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
13302024-09-06T23:37:41.390ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 190486 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:61500 req_id = e70658ff-de86-48eb-9481-81444e5fa3ae response_code = 200 uri = /inventory
13312024-09-06T23:37:41.391ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
13322024-09-06T23:37:41.391ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/ca24db04-f37b-40b1-83c2-23476bec5a4a/config/omicron-zones.json file = common/src/ledger.rs:177
13332024-09-06T23:37:41.391ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/358fc814-3f2a-4b1a-aebb-894434d72d01/config/omicron-zones.json file = common/src/ledger.rs:177
13342024-09-06T23:37:41.391ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 709 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:61500 req_id = f6bef100-1b89-47e3-ae4f-4fe903655b3b response_code = 200 uri = /omicron-zones
13352024-09-06T23:37:42.384ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:46987
13362024-09-06T23:37:42.385ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 93 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:46987 req_id = c406f1bb-9ab0-4927-b5cb-ba3a18f7d9bf response_code = 200 uri = /vpc-routes
13372024-09-06T23:37:42.516ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:52380
13382024-09-06T23:37:42.516ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 161 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:52380 req_id = 4630ac90-4727-4e76-aaf0-e0b10019d309 response_code = 200 uri = /v2p
13392024-09-06T23:37:42.648ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:44775
13402024-09-06T23:37:42.649ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 95 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:44775 req_id = 0f4a7195-54bc-48c1-8fad-8286c25faeff response_code = 200 uri = /vpc-routes
13412024-09-06T23:37:42.659ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 580 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:46987 req_id = f723ae16-0060-497b-b54a-479cfa7d9ecc response_code = 204 uri = /vpc-routes
13422024-09-06T23:37:42.677ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51160
13432024-09-06T23:37:42.677ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 40 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:51160 req_id = db97b7b7-9893-4332-bf88-00ebfbdab53b response_code = 200 uri = /vpc-routes
13442024-09-06T23:37:42.692ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 108 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:51160 req_id = e2c94d08-cd69-443b-88fb-0bbb24d13e68 response_code = 204 uri = /vpc-routes
13452024-09-06T23:37:42.739ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:56187
13462024-09-06T23:37:42.739ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 110 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:56187 req_id = 4781fdae-fcb7-4db0-8c5e-843eea4fbf37 response_code = 200 uri = /v2p
13472024-09-06T23:37:42.773ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 107 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:44775 req_id = 8a3c5328-eb57-4a98-9005-11f543b5cb55 response_code = 204 uri = /vpc-routes
13482024-09-06T23:37:42.776ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:41970
13492024-09-06T23:37:42.777ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 918 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:41970 req_id = 27a2f2bb-9726-41a2-b39c-8ad8a658c64b response_code = 200 uri = /v2p
13502024-09-06T23:37:43.969ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:50915
13512024-09-06T23:37:43.970ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 183029ac-b909-4e34-9803-fdf7533f1a8f, kind: Service { id: 5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6 }, name: Name("external-dns-5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 179, 238])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7ef66645-e8c6-4c80-8f48-af73dfa3e97b, kind: Service { id: 195e4e82-1cc4-4f3b-a893-c1474bd26575 }, name: Name("external-dns-195e4e82-1cc4-4f3b-a893-c1474bd26575"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 212, 58])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 41a43463-6084-4d04-ad1b-f1d1ddccceaf, kind: Service { id: 9ad0f243-aeb1-455d-97ad-7dc9ca9c417c }, name: Name("nexus-9ad0f243-aeb1-455d-97ad-7dc9ca9c417c"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 248, 203])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a0534026-8f14-4794-ba39-35fb89d6d21c, kind: Service { id: cb51aad1-b6ab-44e1-8492-82c028d74c7f }, name: Name("nexus-cb51aad1-b6ab-44e1-8492-82c028d74c7f"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 170, 148])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ce58de05-7c50-42fb-bdcb-23c435be246c, kind: Service { id: 6c363cb4-229c-47a8-9d81-abd1714293f2 }, name: Name("nexus-6c363cb4-229c-47a8-9d81-abd1714293f2"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 218, 98])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
13522024-09-06T23:37:43.970ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
13532024-09-06T23:37:43.970ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
13542024-09-06T23:37:43.970ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
13552024-09-06T23:37:43.970ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
13562024-09-06T23:37:43.970ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
13572024-09-06T23:37:43.970ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
13582024-09-06T23:37:43.970ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 879 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:50915 req_id = fb112c15-2a37-4265-846c-8526cf3b9d5b response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
13592024-09-06T23:37:43.971ZINFOSledAgent (RSS): Handoff to Nexus is complete file = sled-agent/src/rack_setup/service.rs:959
13602024-09-06T23:37:44.206ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:63954
13612024-09-06T23:37:44.207ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 183029ac-b909-4e34-9803-fdf7533f1a8f, kind: Service { id: 5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6 }, name: Name("external-dns-5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 179, 238])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7ef66645-e8c6-4c80-8f48-af73dfa3e97b, kind: Service { id: 195e4e82-1cc4-4f3b-a893-c1474bd26575 }, name: Name("external-dns-195e4e82-1cc4-4f3b-a893-c1474bd26575"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 212, 58])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 41a43463-6084-4d04-ad1b-f1d1ddccceaf, kind: Service { id: 9ad0f243-aeb1-455d-97ad-7dc9ca9c417c }, name: Name("nexus-9ad0f243-aeb1-455d-97ad-7dc9ca9c417c"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 248, 203])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a0534026-8f14-4794-ba39-35fb89d6d21c, kind: Service { id: cb51aad1-b6ab-44e1-8492-82c028d74c7f }, name: Name("nexus-cb51aad1-b6ab-44e1-8492-82c028d74c7f"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 170, 148])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ce58de05-7c50-42fb-bdcb-23c435be246c, kind: Service { id: 6c363cb4-229c-47a8-9d81-abd1714293f2 }, name: Name("nexus-6c363cb4-229c-47a8-9d81-abd1714293f2"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 218, 98])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
13622024-09-06T23:37:44.207ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
13632024-09-06T23:37:44.207ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
13642024-09-06T23:37:44.207ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
13652024-09-06T23:37:44.207ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
13662024-09-06T23:37:44.207ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
13672024-09-06T23:37:44.207ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
13682024-09-06T23:37:44.207ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 266 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:63954 req_id = fb577cdb-1463-44c1-9a8e-db1b2aed6933 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
13692024-09-06T23:37:44.248ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:42590
13702024-09-06T23:37:44.249ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 183029ac-b909-4e34-9803-fdf7533f1a8f, kind: Service { id: 5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6 }, name: Name("external-dns-5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 179, 238])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7ef66645-e8c6-4c80-8f48-af73dfa3e97b, kind: Service { id: 195e4e82-1cc4-4f3b-a893-c1474bd26575 }, name: Name("external-dns-195e4e82-1cc4-4f3b-a893-c1474bd26575"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 212, 58])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 41a43463-6084-4d04-ad1b-f1d1ddccceaf, kind: Service { id: 9ad0f243-aeb1-455d-97ad-7dc9ca9c417c }, name: Name("nexus-9ad0f243-aeb1-455d-97ad-7dc9ca9c417c"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 248, 203])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a0534026-8f14-4794-ba39-35fb89d6d21c, kind: Service { id: cb51aad1-b6ab-44e1-8492-82c028d74c7f }, name: Name("nexus-cb51aad1-b6ab-44e1-8492-82c028d74c7f"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 170, 148])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ce58de05-7c50-42fb-bdcb-23c435be246c, kind: Service { id: 6c363cb4-229c-47a8-9d81-abd1714293f2 }, name: Name("nexus-6c363cb4-229c-47a8-9d81-abd1714293f2"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 218, 98])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
13712024-09-06T23:37:44.249ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
13722024-09-06T23:37:44.249ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
13732024-09-06T23:37:44.249ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
13742024-09-06T23:37:44.249ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
13752024-09-06T23:37:44.249ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
13762024-09-06T23:37:44.249ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
13772024-09-06T23:37:44.249ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 176 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:42590 req_id = 85367966-37ad-4e5e-abd8-1c97e94198cb response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
13782024-09-06T23:37:44.288ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:45691
13792024-09-06T23:37:44.288ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
13802024-09-06T23:37:44.481ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 192726 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:45691 req_id = 29b59448-14f5-42a7-a686-b6a6b5b970d1 response_code = 200 uri = /inventory
13812024-09-06T23:37:44.481ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:35454
13822024-09-06T23:37:44.482ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
13832024-09-06T23:37:44.482ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/ca24db04-f37b-40b1-83c2-23476bec5a4a/config/omicron-zones.json file = common/src/ledger.rs:177
13842024-09-06T23:37:44.482ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 183029ac-b909-4e34-9803-fdf7533f1a8f, kind: Service { id: 5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6 }, name: Name("external-dns-5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 179, 238])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7ef66645-e8c6-4c80-8f48-af73dfa3e97b, kind: Service { id: 195e4e82-1cc4-4f3b-a893-c1474bd26575 }, name: Name("external-dns-195e4e82-1cc4-4f3b-a893-c1474bd26575"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 212, 58])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 41a43463-6084-4d04-ad1b-f1d1ddccceaf, kind: Service { id: 9ad0f243-aeb1-455d-97ad-7dc9ca9c417c }, name: Name("nexus-9ad0f243-aeb1-455d-97ad-7dc9ca9c417c"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 248, 203])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a0534026-8f14-4794-ba39-35fb89d6d21c, kind: Service { id: cb51aad1-b6ab-44e1-8492-82c028d74c7f }, name: Name("nexus-cb51aad1-b6ab-44e1-8492-82c028d74c7f"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 170, 148])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ce58de05-7c50-42fb-bdcb-23c435be246c, kind: Service { id: 6c363cb4-229c-47a8-9d81-abd1714293f2 }, name: Name("nexus-6c363cb4-229c-47a8-9d81-abd1714293f2"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 218, 98])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
13852024-09-06T23:37:44.482ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
13862024-09-06T23:37:44.482ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
13872024-09-06T23:37:44.482ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
13882024-09-06T23:37:44.482ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
13892024-09-06T23:37:44.482ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
13902024-09-06T23:37:44.482ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
13912024-09-06T23:37:44.482ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1020 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:35454 req_id = 8acf6e71-5244-4d72-87fb-a63aa4f6ce82 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
13922024-09-06T23:37:44.483ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/358fc814-3f2a-4b1a-aebb-894434d72d01/config/omicron-zones.json file = common/src/ledger.rs:177
13932024-09-06T23:37:44.483ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1220 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:45691 req_id = 9cfd1cde-2c4d-4937-9446-5bf17cfe68e0 response_code = 200 uri = /omicron-zones
13942024-09-06T23:37:48.715ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:65275
13952024-09-06T23:37:48.715ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 79 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:65275 req_id = a88ff249-5a17-4b59-9ce3-78fe62dbde66 response_code = 200 uri = /vpc-routes
13962024-09-06T23:37:48.728ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 144 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:65275 req_id = 1248bef6-c2a8-4ab5-8baf-74891aca94c4 response_code = 204 uri = /vpc-routes
13972024-09-06T23:37:55.162ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:35763
13982024-09-06T23:37:55.162ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 56 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:35763 req_id = 96d6e77d-d449-4ae0-ba3b-c80f5722d073 response_code = 200 uri = /baseboard
13992024-09-06T23:37:55.202ZINFOSledAgent (dropshot): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:39558 remote_addr = [fd00:1122:3344:101::d]:50080 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c
14002024-09-06T23:37:55.217ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 15850 local_addr = [fd00:1122:3344:101::1]:39558 method = GET remote_addr = [fd00:1122:3344:101::d]:50080 req_id = 18079406-5a7a-44a4-9187-52cfb796d649 response_code = 200 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c uri = /55267d89-b780-4916-86a8-64533b7bfd1c
14012024-09-06T23:38:12.379ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:56389
14022024-09-06T23:38:12.379ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 188 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:56389 req_id = 6f64ae72-8550-4614-8210-e9257dc7f7db response_code = 200 uri = /v2p
14032024-09-06T23:38:12.606ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:44757
14042024-09-06T23:38:12.607ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1109 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:44757 req_id = 50ac790b-324a-446f-a5db-aa06ee0771e8 response_code = 200 uri = /vpc-routes
14052024-09-06T23:38:12.627ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 114 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:44757 req_id = 49afe93a-a067-47cf-a1b6-5f63ca5a594e response_code = 204 uri = /vpc-routes
14062024-09-06T23:38:12.724ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51013
14072024-09-06T23:38:12.725ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1031 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:51013 req_id = 53c6fe08-3daf-44fd-b1fa-bcc763209704 response_code = 200 uri = /vpc-routes
14082024-09-06T23:38:12.751ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 145 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:51013 req_id = 9979d201-bad4-4bdc-9825-1fd45da72242 response_code = 204 uri = /vpc-routes
14092024-09-06T23:38:12.763ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:64783
14102024-09-06T23:38:12.764ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1069 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:64783 req_id = 53f9fb36-4699-4b0f-93f5-a1cfacddd1ba response_code = 200 uri = /vpc-routes
14112024-09-06T23:38:12.775ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 98 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:64783 req_id = 3c36dffb-24ad-41b6-b92c-676e3c0beaf8 response_code = 204 uri = /vpc-routes
14122024-09-06T23:38:12.857ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:42458
14132024-09-06T23:38:12.857ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4014
14142024-09-06T23:38:12.900ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4038
14152024-09-06T23:38:12.954ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 97087 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:42458 req_id = 71e8b0b4-505f-402b-9d1e-be9531fc64e9 response_code = 204 uri = /switch-ports
14162024-09-06T23:38:12.954ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:35973
14172024-09-06T23:38:12.954ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 173 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:35973 req_id = c65b1ba1-f58d-4d21-af91-9d224c8dc7f1 response_code = 200 uri = /v2p
14182024-09-06T23:38:12.955ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1208 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:42458 req_id = c32a081b-67a3-4de8-b632-1f1bbda4b267 response_code = 200 uri = /network-bootstore-config
14192024-09-06T23:38:12.998ZINFOSledAgent (bootstore): Attempting to update network config with generation: 2, current_generation: 1 file = bootstore/src/schemes/v0/peer.rs:564 peer_id = pc-sock-i86pc
14202024-09-06T23:38:12.998ZINFOSledAgent (bootstore): Writing ledger to /pool/int/ca24db04-f37b-40b1-83c2-23476bec5a4a/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
14212024-09-06T23:38:12.999ZINFOSledAgent (bootstore): Writing ledger to /pool/int/358fc814-3f2a-4b1a-aebb-894434d72d01/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
14222024-09-06T23:38:13.000ZINFOSledAgent (bootstore): Broadcasting network config with generation 2 file = bootstore/src/schemes/v0/peer.rs:648 peer_id = pc-sock-i86pc
14232024-09-06T23:38:13.000ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3350 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:42458 req_id = 7597c344-021c-473c-8881-3528a889c871 response_code = 204 uri = /network-bootstore-config
14242024-09-06T23:38:13.029ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:54329
14252024-09-06T23:38:13.029ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4014
14262024-09-06T23:38:13.066ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4038
14272024-09-06T23:38:13.120ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 91153 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:54329 req_id = 7739218d-55f4-43e2-ba9b-4e3df7789c93 response_code = 204 uri = /switch-ports
14282024-09-06T23:38:13.120ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:49819
14292024-09-06T23:38:13.120ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 126 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:49819 req_id = 2a4edd9a-72a9-47bf-b121-94edd0391d6e response_code = 200 uri = /v2p
14302024-09-06T23:38:13.121ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 101 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:54329 req_id = d02c86b6-258f-476f-9efc-b244c31c8bd7 response_code = 200 uri = /network-bootstore-config
14312024-09-06T23:38:13.281ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:36849
14322024-09-06T23:38:13.281ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4014
14332024-09-06T23:38:13.314ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4038
14342024-09-06T23:38:13.371ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 90291 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:36849 req_id = 88fc4a41-5c66-4b00-a579-688ddf97cdb7 response_code = 204 uri = /switch-ports
14352024-09-06T23:38:13.372ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 859 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:36849 req_id = 067f6d91-0d88-48a0-8942-e36c04231f66 response_code = 200 uri = /network-bootstore-config
14362024-09-06T23:38:25.200ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:64363
14372024-09-06T23:38:25.200ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 40 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:64363 req_id = ed92f0e2-70a1-4061-8de8-d09b8dc6bd1a response_code = 200 uri = /baseboard
14382024-09-06T23:38:25.206ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 4881 local_addr = [fd00:1122:3344:101::1]:39558 method = GET remote_addr = [fd00:1122:3344:101::d]:50080 req_id = eeaa5b06-7716-4c0d-958b-301839736b9b response_code = 200 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c uri = /55267d89-b780-4916-86a8-64533b7bfd1c
14392024-09-06T23:38:42.422ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:49559
14402024-09-06T23:38:42.422ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 113 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:49559 req_id = 8a017230-3d95-4476-9af8-211130773011 response_code = 200 uri = /vpc-routes
14412024-09-06T23:38:42.444ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 162 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:49559 req_id = d63e56a6-cccb-44e9-b23a-8e0899c4c070 response_code = 204 uri = /vpc-routes
14422024-09-06T23:38:42.509ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:34981
14432024-09-06T23:38:42.509ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 198 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:34981 req_id = 6464f309-14c8-4f94-a1ee-bf82c9618a8e response_code = 200 uri = /v2p
14442024-09-06T23:38:42.547ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:60905
14452024-09-06T23:38:42.549ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4014
14462024-09-06T23:38:42.595ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4038
14472024-09-06T23:38:42.659ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 112218 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:60905 req_id = bd49111d-5415-43e7-bb6f-2acc7995b719 response_code = 204 uri = /switch-ports
14482024-09-06T23:38:42.660ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:55649
14492024-09-06T23:38:42.660ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:34503
14502024-09-06T23:38:42.660ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 94 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:55649 req_id = 5af605d1-629d-4693-9d7a-8e143f04599b response_code = 200 uri = /vpc-routes
14512024-09-06T23:38:42.660ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 108 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:34503 req_id = dd7b27e7-7942-4184-9f5a-e102000afb11 response_code = 200 uri = /vpc-routes
14522024-09-06T23:38:42.660ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 227 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:60905 req_id = 4b8b9cbc-701c-4a6b-b252-b3f81238b1c8 response_code = 200 uri = /network-bootstore-config
14532024-09-06T23:38:42.678ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 177 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:55649 req_id = ff98aed9-95ad-4a1f-a5d2-1896c86c618b response_code = 204 uri = /vpc-routes
14542024-09-06T23:38:42.682ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 105 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:34503 req_id = f66b68fc-9adc-4b66-977f-7fdd4f10d254 response_code = 204 uri = /vpc-routes
14552024-09-06T23:38:42.715ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:46000
14562024-09-06T23:38:42.715ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4014
14572024-09-06T23:38:42.757ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4038
14582024-09-06T23:38:42.814ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 98847 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:46000 req_id = 77d2bec3-df80-4b66-bf4c-19c204196a80 response_code = 204 uri = /switch-ports
14592024-09-06T23:38:42.814ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:34442
14602024-09-06T23:38:42.815ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 994 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:34442 req_id = 9ff36ab7-5842-4cfd-8608-fdfa36f386b6 response_code = 200 uri = /v2p
14612024-09-06T23:38:42.815ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1032 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:46000 req_id = e2fead33-e93f-4d9c-8659-f55f8a3d0c19 response_code = 200 uri = /network-bootstore-config
14622024-09-06T23:38:42.818ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:53798
14632024-09-06T23:38:42.818ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4014
14642024-09-06T23:38:42.840ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:49478
14652024-09-06T23:38:42.840ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 169 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:49478 req_id = b622ceff-658a-4234-9d24-02f89bf56425 response_code = 200 uri = /v2p
14662024-09-06T23:38:42.858ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4038
14672024-09-06T23:38:42.915ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 97044 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:53798 req_id = a96eff33-3ec5-4c09-82e8-f697ea985e06 response_code = 204 uri = /switch-ports
14682024-09-06T23:38:42.915ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 138 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:53798 req_id = 7af9a77e-810b-4b8e-abf7-822c71ba0869 response_code = 200 uri = /network-bootstore-config
14692024-09-06T23:38:55.208ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 6758 local_addr = [fd00:1122:3344:101::1]:39558 method = GET remote_addr = [fd00:1122:3344:101::d]:50080 req_id = 9ae7fae0-5e2e-4a65-b333-cb979c345686 response_code = 200 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c uri = /55267d89-b780-4916-86a8-64533b7bfd1c
14702024-09-06T23:38:55.233ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:35521
14712024-09-06T23:38:55.233ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 56 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:35521 req_id = 31f3f9d6-dc58-4c63-b167-db00b6098f8c response_code = 200 uri = /baseboard
14722024-09-06T23:39:12.479ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:61215
14732024-09-06T23:39:12.479ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 131 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:61215 req_id = 3aa5d02b-9abf-4da0-b3a0-4078865330cd response_code = 200 uri = /vpc-routes
14742024-09-06T23:39:12.749ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:41313
14752024-09-06T23:39:12.749ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 216 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:41313 req_id = 2013d4fd-1e30-48d0-b929-312cd7e5a445 response_code = 200 uri = /v2p
14762024-09-06T23:39:12.761ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 129 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:61215 req_id = 6ec81be6-dd9e-43de-93c8-1239437fa945 response_code = 204 uri = /vpc-routes
14772024-09-06T23:39:12.833ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:48043
14782024-09-06T23:39:12.833ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 35 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:48043 req_id = 789cf32d-4e3e-409e-a06b-c5c1eac8a56f response_code = 200 uri = /vpc-routes
14792024-09-06T23:39:12.844ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 126 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:48043 req_id = d62d679d-1475-4045-b560-27bf1ff3dc9c response_code = 204 uri = /vpc-routes
14802024-09-06T23:39:12.854ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:37166
14812024-09-06T23:39:12.854ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4014
14822024-09-06T23:39:12.896ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4038
14832024-09-06T23:39:12.954ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 99562 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:37166 req_id = 4512cda2-f7c4-457f-b0d1-3914b745b4df response_code = 204 uri = /switch-ports
14842024-09-06T23:39:12.954ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:50725
14852024-09-06T23:39:12.954ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 225 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:50725 req_id = c3cb36b2-af09-45fa-8141-1cc1c50cf481 response_code = 200 uri = /v2p
14862024-09-06T23:39:12.955ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:40674
14872024-09-06T23:39:12.955ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 108 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:40674 req_id = d07a6790-1819-4f77-b83f-55611c3d108b response_code = 200 uri = /vpc-routes
14882024-09-06T23:39:12.955ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1160 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:37166 req_id = 23341cee-0a65-49c2-ad41-00538ce68ded response_code = 200 uri = /network-bootstore-config
14892024-09-06T23:39:12.968ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 128 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:40674 req_id = a6c60eb7-4a26-4d63-8fb8-e4c6793c4a79 response_code = 204 uri = /vpc-routes
14902024-09-06T23:39:13.037ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:53855
14912024-09-06T23:39:13.037ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4014
14922024-09-06T23:39:13.073ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4038
14932024-09-06T23:39:13.125ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 87700 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:53855 req_id = 1f3c6311-34d8-417f-b996-0ab94921835c response_code = 204 uri = /switch-ports
14942024-09-06T23:39:13.125ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:64324
14952024-09-06T23:39:13.125ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:60932
14962024-09-06T23:39:13.125ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4014
14972024-09-06T23:39:13.129ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3514 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:53855 req_id = ecb369f8-fb38-4c91-8dff-140e9ad48e74 response_code = 200 uri = /network-bootstore-config
14982024-09-06T23:39:13.129ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3537 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:60932 req_id = 02e51d1a-2e7f-452b-b052-4694c8faca43 response_code = 200 uri = /v2p
14992024-09-06T23:39:13.159ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4038
15002024-09-06T23:39:13.211ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 85554 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:64324 req_id = 3d363ab0-856b-4b9d-8a22-240904667bff response_code = 204 uri = /switch-ports
15012024-09-06T23:39:13.211ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 166 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:64324 req_id = c103f7e9-4e2c-4aab-8f31-741afcf36ff4 response_code = 200 uri = /network-bootstore-config
15022024-09-06T23:39:25.207ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 5505 local_addr = [fd00:1122:3344:101::1]:39558 method = GET remote_addr = [fd00:1122:3344:101::d]:50080 req_id = 9fd85521-b24a-4f09-81d5-08c0446e6635 response_code = 200 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c uri = /55267d89-b780-4916-86a8-64533b7bfd1c
15032024-09-06T23:39:25.267ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:64262
15042024-09-06T23:39:25.267ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 39 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:64262 req_id = 78f27141-8850-4d10-afeb-26ae9b5c4cf9 response_code = 200 uri = /baseboard
15052024-09-06T23:39:42.500ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:58629
15062024-09-06T23:39:42.500ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 113 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:58629 req_id = d1581ec0-7d91-487b-8a96-f2a5af456bf8 response_code = 200 uri = /vpc-routes
15072024-09-06T23:39:42.506ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:45033
15082024-09-06T23:39:42.506ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 53 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:45033 req_id = 09371ab0-7595-4b8e-bf9a-be1381f44eac response_code = 200 uri = /vpc-routes
15092024-09-06T23:39:42.526ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2159 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:58629 req_id = c63671b7-f6e4-4551-bf8d-64b8bdb2508d response_code = 204 uri = /vpc-routes
15102024-09-06T23:39:42.545ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 109 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:45033 req_id = 3db3623c-429d-4768-b224-f17bff9d4406 response_code = 204 uri = /vpc-routes
15112024-09-06T23:39:42.587ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:59359
15122024-09-06T23:39:42.588ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 179 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:59359 req_id = ebbf426e-6427-4971-b2b9-29baa76859ae response_code = 200 uri = /v2p
15132024-09-06T23:39:42.627ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:44293
15142024-09-06T23:39:42.627ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4014
15152024-09-06T23:39:42.672ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4038
15162024-09-06T23:39:42.743ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 115801 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:44293 req_id = fad777f6-d6ab-4e72-9795-c7892c1a9e94 response_code = 204 uri = /switch-ports
15172024-09-06T23:39:42.743ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:63649
15182024-09-06T23:39:42.743ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4014
15192024-09-06T23:39:42.776ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4038
15202024-09-06T23:39:42.832ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 89286 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:63649 req_id = 90e534dc-258d-4664-8e38-016a40e1f8d3 response_code = 204 uri = /switch-ports
15212024-09-06T23:39:42.833ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:50651
15222024-09-06T23:39:42.833ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 203 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:44293 req_id = edc923e4-51e5-4587-9d7f-508437d4f6eb response_code = 200 uri = /network-bootstore-config
15232024-09-06T23:39:42.833ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:46225
15242024-09-06T23:39:42.833ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 226 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:63649 req_id = 2c8c858d-9512-4eaa-b0c7-465ae6ad0e27 response_code = 200 uri = /network-bootstore-config
15252024-09-06T23:39:42.834ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 926 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:50651 req_id = 8d9aeb8f-6b49-4d8e-b7e5-1cd31f94ec15 response_code = 200 uri = /vpc-routes
15262024-09-06T23:39:42.834ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 991 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:46225 req_id = aaefd60e-316f-4aaa-a822-f23ac13e82b6 response_code = 200 uri = /v2p
15272024-09-06T23:39:42.844ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 112 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:50651 req_id = 7522c0b1-2b42-4909-9802-f91994cb89d9 response_code = 204 uri = /vpc-routes
15282024-09-06T23:39:42.867ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:40166
15292024-09-06T23:39:42.867ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 121 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:40166 req_id = 9d55c91c-2fdd-428f-93c0-a6f2c1c69192 response_code = 200 uri = /v2p
15302024-09-06T23:39:42.899ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:53326
15312024-09-06T23:39:42.899ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4014
15322024-09-06T23:39:42.935ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4038
15332024-09-06T23:39:42.984ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 85700 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:53326 req_id = 4614ea95-a4ee-4316-a7cd-5aeac444c89b response_code = 204 uri = /switch-ports
15342024-09-06T23:39:42.985ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 106 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:53326 req_id = ba672ce0-d466-4550-8531-16155358ab46 response_code = 200 uri = /network-bootstore-config
15352024-09-06T23:39:55.217ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 5045 local_addr = [fd00:1122:3344:101::1]:39558 method = GET remote_addr = [fd00:1122:3344:101::d]:50080 req_id = e2e6b83e-9cb0-466f-891e-7f2a3a0db9da response_code = 200 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c uri = /55267d89-b780-4916-86a8-64533b7bfd1c
15362024-09-06T23:39:55.300ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:46193
15372024-09-06T23:39:55.300ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 38 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:46193 req_id = 1788ca01-69c5-4c1c-ab6d-889aabdebda5 response_code = 200 uri = /baseboard
15382024-09-06T23:40:12.589ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:37074
15392024-09-06T23:40:12.589ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 65 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:37074 req_id = 7bae0827-ed53-494f-a4d9-57f1397a8141 response_code = 200 uri = /vpc-routes
15402024-09-06T23:40:12.636ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 153 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:37074 req_id = 653b550e-df5d-4900-8e6a-83983338052b response_code = 204 uri = /vpc-routes
15412024-09-06T23:40:12.689ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:38582
15422024-09-06T23:40:12.689ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 49 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:38582 req_id = f3ed5d5d-94dd-43ac-9109-a657127a16c7 response_code = 200 uri = /vpc-routes
15432024-09-06T23:40:12.704ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 118 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:38582 req_id = f6ba5e81-a98f-4f89-abfe-eb81c7bb40ce response_code = 204 uri = /vpc-routes
15442024-09-06T23:40:12.773ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:45432
15452024-09-06T23:40:12.773ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 154 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:45432 req_id = f7745f4b-fa49-4f8f-adf2-72e84b60bfd3 response_code = 200 uri = /v2p
15462024-09-06T23:40:12.823ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:39441
15472024-09-06T23:40:12.823ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4014
15482024-09-06T23:40:12.872ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4038
15492024-09-06T23:40:12.934ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 110396 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:39441 req_id = c0e0c2e4-c5ae-43bd-8e57-ff892263c180 response_code = 204 uri = /switch-ports
15502024-09-06T23:40:12.934ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:50453
15512024-09-06T23:40:12.934ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 87 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:50453 req_id = 80ba5a90-eed5-43f6-b7b6-220146f778d4 response_code = 200 uri = /vpc-routes
15522024-09-06T23:40:12.934ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 248 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:39441 req_id = d4bc5105-71d7-4563-a9af-fb2de655df6c response_code = 200 uri = /network-bootstore-config
15532024-09-06T23:40:12.946ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:55491
15542024-09-06T23:40:12.946ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 202 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:55491 req_id = b678b27b-fdda-480b-adb4-07132303a9c9 response_code = 200 uri = /v2p
15552024-09-06T23:40:12.947ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 114 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:50453 req_id = ed0d5d46-491f-47f2-96c3-b70838fb1c83 response_code = 204 uri = /vpc-routes
15562024-09-06T23:40:12.993ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:35793
15572024-09-06T23:40:12.993ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 101 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:35793 req_id = c242d488-f4e7-48cc-a85b-09db8334a645 response_code = 200 uri = /v2p
15582024-09-06T23:40:13.042ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:60372
15592024-09-06T23:40:13.042ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4014
15602024-09-06T23:40:13.079ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4038
15612024-09-06T23:40:13.131ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 88564 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:60372 req_id = b3ca6b8a-8baa-4434-bbc5-3b9eba04619e response_code = 204 uri = /switch-ports
15622024-09-06T23:40:13.131ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:54057
15632024-09-06T23:40:13.131ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4014
15642024-09-06T23:40:13.163ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4038
15652024-09-06T23:40:13.209ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 77513 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:54057 req_id = 2ae0915f-481b-416a-9d20-e3ce451de270 response_code = 204 uri = /switch-ports
15662024-09-06T23:40:13.209ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 97 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:60372 req_id = e39b1d88-4557-44bc-8a61-5bf3bf15dedc response_code = 200 uri = /network-bootstore-config
15672024-09-06T23:40:13.210ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 764 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:54057 req_id = 2b9005ca-0bff-4e24-abbf-336209ca8d33 response_code = 200 uri = /network-bootstore-config
15682024-09-06T23:40:14.911ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
15692024-09-06T23:40:25.292ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 6981 local_addr = [fd00:1122:3344:101::1]:39558 method = GET remote_addr = [fd00:1122:3344:101::d]:50080 req_id = c7168556-b127-47ed-91c3-a523edc60b56 response_code = 200 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c uri = /55267d89-b780-4916-86a8-64533b7bfd1c
15702024-09-06T23:40:25.332ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:46088
15712024-09-06T23:40:25.333ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 48 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:46088 req_id = c6d60a2a-a62f-4eb5-9025-1438e98edcf6 response_code = 200 uri = /baseboard
15722024-09-06T23:40:42.424ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:38351
15732024-09-06T23:40:42.424ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 99 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:38351 req_id = 6745264c-5932-4054-80c6-d4d27b608c8d response_code = 200 uri = /vpc-routes
15742024-09-06T23:40:42.438ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 163 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:38351 req_id = 32b38fc6-d5c5-49d4-b025-559641908aa9 response_code = 204 uri = /vpc-routes
15752024-09-06T23:40:42.568ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:35439
15762024-09-06T23:40:42.568ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4014
15772024-09-06T23:40:42.615ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4038
15782024-09-06T23:40:42.678ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 109809 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:35439 req_id = 83cf4173-82b7-4701-9af7-9cdadd4574a4 response_code = 204 uri = /switch-ports
15792024-09-06T23:40:42.678ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:46392
15802024-09-06T23:40:42.682ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3622 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:46392 req_id = 08b14a58-6533-47f4-84e2-531c5d5caf8e response_code = 200 uri = /v2p
15812024-09-06T23:40:42.682ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3734 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:35439 req_id = d8a4f19f-5378-441c-b17c-8f0141fd97d1 response_code = 200 uri = /network-bootstore-config
15822024-09-06T23:40:42.697ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:43100
15832024-09-06T23:40:42.697ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 46 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:43100 req_id = dd96e3e5-18e7-48de-9fac-5ed48a109302 response_code = 200 uri = /vpc-routes
15842024-09-06T23:40:42.710ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 105 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:43100 req_id = ae538e2c-dc22-4639-9b54-59b3574959b7 response_code = 204 uri = /vpc-routes
15852024-09-06T23:40:42.734ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:37036
15862024-09-06T23:40:42.734ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 715 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:37036 req_id = e73cd2e1-f1f9-4676-832a-17829351b40e response_code = 200 uri = /v2p
15872024-09-06T23:40:42.764ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:42073
15882024-09-06T23:40:42.764ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 65 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:42073 req_id = caee5cbe-1dce-464d-8bab-6acebb2d5988 response_code = 200 uri = /vpc-routes
15892024-09-06T23:40:42.781ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 146 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:42073 req_id = 658439f3-222f-45e1-a1dc-2b94bf4103a5 response_code = 204 uri = /vpc-routes
15902024-09-06T23:40:42.824ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:60564
15912024-09-06T23:40:42.824ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4014
15922024-09-06T23:40:42.861ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4038
15932024-09-06T23:40:42.922ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 97787 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:60564 req_id = 93de0324-1bd7-404a-b0d4-9f5f33d96951 response_code = 204 uri = /switch-ports
15942024-09-06T23:40:42.922ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:43101
15952024-09-06T23:40:42.922ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4014
15962024-09-06T23:40:42.957ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4038
15972024-09-06T23:40:43.012ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 89229 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:43101 req_id = 70fa134d-e841-4467-8fda-b14add7fc601 response_code = 204 uri = /switch-ports
15982024-09-06T23:40:43.012ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:43277
15992024-09-06T23:40:43.013ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 954 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:43277 req_id = e477eed2-4bad-42a0-9547-f25659014e26 response_code = 200 uri = /v2p
16002024-09-06T23:40:43.013ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 976 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:43101 req_id = 83783f44-768d-4547-9410-c7c324a521ea response_code = 200 uri = /network-bootstore-config
16012024-09-06T23:40:43.013ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1039 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:60564 req_id = c5d10772-285a-49f5-8075-867bb1a89bee response_code = 200 uri = /network-bootstore-config
16022024-09-06T23:40:55.291ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 5053 local_addr = [fd00:1122:3344:101::1]:39558 method = GET remote_addr = [fd00:1122:3344:101::d]:50080 req_id = 2184ba3e-761c-4dde-87c7-3be7653f924b response_code = 200 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c uri = /55267d89-b780-4916-86a8-64533b7bfd1c
16032024-09-06T23:40:55.368ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:53704
16042024-09-06T23:40:55.368ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 45 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:53704 req_id = ce0f16f6-0c28-4f37-ada1-ea07d00abae4 response_code = 200 uri = /baseboard
16052024-09-06T23:41:12.547ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:36036
16062024-09-06T23:41:12.547ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 150 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:36036 req_id = f6f3bdf8-e63a-4dba-99d1-4d613f579de8 response_code = 200 uri = /vpc-routes
16072024-09-06T23:41:12.587ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1477 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:36036 req_id = 5534629d-5e16-4b92-843a-760b90251a4c response_code = 204 uri = /vpc-routes
16082024-09-06T23:41:12.682ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:60761
16092024-09-06T23:41:12.682ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 142 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:60761 req_id = 3e37ace9-3874-4801-b87c-67ddf32cefff response_code = 200 uri = /v2p
16102024-09-06T23:41:12.740ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:59009
16112024-09-06T23:41:12.740ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4014
16122024-09-06T23:41:12.781ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4038
16132024-09-06T23:41:12.842ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 102067 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:59009 req_id = cecf6372-6e5c-485d-869d-bac77ccc75fd response_code = 204 uri = /switch-ports
16142024-09-06T23:41:12.843ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:64641
16152024-09-06T23:41:12.843ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 175 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:59009 req_id = c3ad9a30-46ad-4d9e-a3e7-f17f98ed183c response_code = 200 uri = /network-bootstore-config
16162024-09-06T23:41:12.843ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 123 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:64641 req_id = 2c7e3e23-97b8-4179-a33c-65fbd57cad01 response_code = 200 uri = /vpc-routes
16172024-09-06T23:41:12.853ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 175 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:64641 req_id = 3df29b87-2592-4fd0-af61-23001640efaf response_code = 204 uri = /vpc-routes
16182024-09-06T23:41:12.883ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:45001
16192024-09-06T23:41:12.883ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 139 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:45001 req_id = e894b0ca-1ba3-41ec-ac3c-81870cc9d1f2 response_code = 200 uri = /v2p
16202024-09-06T23:41:12.928ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:42610
16212024-09-06T23:41:12.928ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 65 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:42610 req_id = 2511f4aa-f990-40e5-9879-6dcc5f8ed458 response_code = 200 uri = /vpc-routes
16222024-09-06T23:41:12.943ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 124 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:42610 req_id = eff43ce2-8160-48f2-9ca9-f6128b11949f response_code = 204 uri = /vpc-routes
16232024-09-06T23:41:12.955ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:55758
16242024-09-06T23:41:12.955ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4014
16252024-09-06T23:41:12.995ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4038
16262024-09-06T23:41:13.050ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 95094 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:55758 req_id = 28ab661b-00e7-4622-8be7-ab6e68307752 response_code = 204 uri = /switch-ports
16272024-09-06T23:41:13.057ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 734 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:55758 req_id = ebc29f9a-e335-47df-9972-98a0ee640608 response_code = 200 uri = /network-bootstore-config
16282024-09-06T23:41:13.062ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:45802
16292024-09-06T23:41:13.062ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4014
16302024-09-06T23:41:13.095ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4038
16312024-09-06T23:41:13.145ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 82933 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:45802 req_id = 5473e05a-a50e-4a56-b75d-d1e69bf088ac response_code = 204 uri = /switch-ports
16322024-09-06T23:41:13.145ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:60701
16332024-09-06T23:41:13.145ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 128 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:45802 req_id = 5b9a543f-7fcd-459c-8250-f07467ab169a response_code = 200 uri = /network-bootstore-config
16342024-09-06T23:41:13.146ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 950 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:60701 req_id = 29580139-1945-4a9a-9f8e-a932b3b55c8d response_code = 200 uri = /v2p
16352024-09-06T23:41:25.291ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 5299 local_addr = [fd00:1122:3344:101::1]:39558 method = GET remote_addr = [fd00:1122:3344:101::d]:50080 req_id = a1fec937-66f6-47d6-9af0-29fb217e6f68 response_code = 200 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c uri = /55267d89-b780-4916-86a8-64533b7bfd1c
16362024-09-06T23:41:25.402ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:49243
16372024-09-06T23:41:25.402ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 39 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:49243 req_id = aa545dba-f951-45e7-86c6-8199dcc0c7d0 response_code = 200 uri = /baseboard
16382024-09-06T23:41:42.479ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:47106
16392024-09-06T23:41:42.479ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 161 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:47106 req_id = fad07179-ac3c-4ab9-8045-6c52c5c43bee response_code = 200 uri = /vpc-routes
16402024-09-06T23:41:42.519ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:64091
16412024-09-06T23:41:42.520ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1361 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:64091 req_id = bcba25a6-2617-4613-9576-cc87d1623219 response_code = 200 uri = /v2p
16422024-09-06T23:41:42.535ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 238 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:47106 req_id = f5cb4d39-dab8-4cae-a5fb-772441b0e201 response_code = 204 uri = /vpc-routes
16432024-09-06T23:41:42.593ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:40422
16442024-09-06T23:41:42.593ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4014
16452024-09-06T23:41:42.639ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4038
16462024-09-06T23:41:42.701ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 107346 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:40422 req_id = 044da5dc-03f5-404b-8a47-22b6475f0a51 response_code = 204 uri = /switch-ports
16472024-09-06T23:41:42.701ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:63226
16482024-09-06T23:41:42.701ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:48237
16492024-09-06T23:41:42.701ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 220 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:63226 req_id = 3a7782ea-a1f1-4ba3-9e61-f550a6988c81 response_code = 200 uri = /vpc-routes
16502024-09-06T23:41:42.702ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1233 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:48237 req_id = 911f7cd8-d4b3-4103-a5a6-6d4a6b9a38d9 response_code = 200 uri = /vpc-routes
16512024-09-06T23:41:42.702ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1426 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:40422 req_id = ca5a051d-300a-4430-9c62-ecfefcfcdadf response_code = 200 uri = /network-bootstore-config
16522024-09-06T23:41:42.714ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 129 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:63226 req_id = 01855896-1f68-4751-8585-6b36fb0c2385 response_code = 204 uri = /vpc-routes
16532024-09-06T23:41:42.715ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:56573
16542024-09-06T23:41:42.715ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 105 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:56573 req_id = 4d61db1b-6481-4450-8192-8b1ab3c591aa response_code = 200 uri = /v2p
16552024-09-06T23:41:42.724ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 58 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:48237 req_id = c6926da5-5104-452b-bd20-b4faf53f36ac response_code = 204 uri = /vpc-routes
16562024-09-06T23:41:42.751ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:46109
16572024-09-06T23:41:42.751ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4014
16582024-09-06T23:41:42.788ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4038
16592024-09-06T23:41:42.843ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 92124 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:46109 req_id = b5421cad-12fb-4ff0-9386-eedb2bfb4222 response_code = 204 uri = /switch-ports
16602024-09-06T23:41:42.843ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:44472
16612024-09-06T23:41:42.843ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:48071
16622024-09-06T23:41:42.844ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4014
16632024-09-06T23:41:42.847ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3585 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:48071 req_id = f51d39ff-6b2a-4ea0-b42d-e4f87ad27b51 response_code = 200 uri = /v2p
16642024-09-06T23:41:42.847ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 181 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:46109 req_id = 8601b228-a456-4ada-84e1-251c2c5a6042 response_code = 200 uri = /network-bootstore-config
16652024-09-06T23:41:42.880ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4038
16662024-09-06T23:41:42.934ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 90388 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:44472 req_id = 24806904-d177-4d73-836f-3edda91616cf response_code = 204 uri = /switch-ports
16672024-09-06T23:41:42.934ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 144 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:44472 req_id = 7f5533e7-7004-40b1-98fd-6a38dac3602c response_code = 200 uri = /network-bootstore-config
16682024-09-06T23:41:55.294ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 7197 local_addr = [fd00:1122:3344:101::1]:39558 method = GET remote_addr = [fd00:1122:3344:101::d]:50080 req_id = a7f3df78-9e83-4462-93b5-3014936eec06 response_code = 200 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c uri = /55267d89-b780-4916-86a8-64533b7bfd1c
16692024-09-06T23:41:55.441ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:54559
16702024-09-06T23:41:55.441ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 65 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:54559 req_id = 3d543846-9f30-49f8-b641-a2d99b553025 response_code = 200 uri = /baseboard
16712024-09-06T23:42:12.516ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:35847
16722024-09-06T23:42:12.516ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 146 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:35847 req_id = ff83262a-651a-466d-bf14-56e45441431a response_code = 200 uri = /vpc-routes
16732024-09-06T23:42:12.550ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 936 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:35847 req_id = 40e6542d-2ed8-4096-a948-6f024ee84b01 response_code = 204 uri = /vpc-routes
16742024-09-06T23:42:12.743ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:56474
16752024-09-06T23:42:12.743ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 134 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:56474 req_id = 2ced53e7-3f2b-478b-bfc1-8adbc77d8cb5 response_code = 200 uri = /v2p
16762024-09-06T23:42:12.807ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:49343
16772024-09-06T23:42:12.807ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4014
16782024-09-06T23:42:12.854ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4038
16792024-09-06T23:42:12.944ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 137302 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:49343 req_id = 3438b3e1-30f4-4395-8cac-1251f3b05a46 response_code = 204 uri = /switch-ports
16802024-09-06T23:42:12.944ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:35954
16812024-09-06T23:42:12.945ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:50624
16822024-09-06T23:42:12.945ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:42243
16832024-09-06T23:42:12.950ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 5187 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:49343 req_id = ea274fe1-fe87-4b14-a7dd-bd05be08a126 response_code = 200 uri = /network-bootstore-config
16842024-09-06T23:42:12.950ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 5209 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:50624 req_id = 17c6d9ec-00c8-426e-97a6-ee15cf3b59f3 response_code = 200 uri = /vpc-routes
16852024-09-06T23:42:12.950ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 183029ac-b909-4e34-9803-fdf7533f1a8f, kind: Service { id: 5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6 }, name: Name("external-dns-5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 179, 238])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7ef66645-e8c6-4c80-8f48-af73dfa3e97b, kind: Service { id: 195e4e82-1cc4-4f3b-a893-c1474bd26575 }, name: Name("external-dns-195e4e82-1cc4-4f3b-a893-c1474bd26575"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 212, 58])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 41a43463-6084-4d04-ad1b-f1d1ddccceaf, kind: Service { id: 9ad0f243-aeb1-455d-97ad-7dc9ca9c417c }, name: Name("nexus-9ad0f243-aeb1-455d-97ad-7dc9ca9c417c"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 248, 203])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a0534026-8f14-4794-ba39-35fb89d6d21c, kind: Service { id: cb51aad1-b6ab-44e1-8492-82c028d74c7f }, name: Name("nexus-cb51aad1-b6ab-44e1-8492-82c028d74c7f"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 170, 148])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ce58de05-7c50-42fb-bdcb-23c435be246c, kind: Service { id: 6c363cb4-229c-47a8-9d81-abd1714293f2 }, name: Name("nexus-6c363cb4-229c-47a8-9d81-abd1714293f2"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 218, 98])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
16862024-09-06T23:42:12.950ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
16872024-09-06T23:42:12.950ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
16882024-09-06T23:42:12.950ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
16892024-09-06T23:42:12.950ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
16902024-09-06T23:42:12.950ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
16912024-09-06T23:42:12.950ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
16922024-09-06T23:42:12.950ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1368 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:35954 req_id = 87e1ffef-1efd-4528-ae13-b62c5a336aa6 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
16932024-09-06T23:42:12.951ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1602 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:42243 req_id = 089b5979-6ced-4346-9133-b55fa6ff1a19 response_code = 200 uri = /vpc-routes
16942024-09-06T23:42:12.968ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 146 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:42243 req_id = 43c83031-87bb-45f6-a8d7-6635b4728029 response_code = 204 uri = /vpc-routes
16952024-09-06T23:42:12.981ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:51603
16962024-09-06T23:42:12.981ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4014
16972024-09-06T23:42:13.023ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4038
16982024-09-06T23:42:13.078ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 97041 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:51603 req_id = f5942173-78f6-46d3-b667-70ac4ed78c58 response_code = 204 uri = /switch-ports
16992024-09-06T23:42:13.079ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:61587
17002024-09-06T23:42:13.079ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:51808
17012024-09-06T23:42:13.079ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:34232
17022024-09-06T23:42:13.079ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:48315
17032024-09-06T23:42:13.079ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 397 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:51603 req_id = b4902cfa-bfe6-42a1-8204-3d87be3f2e95 response_code = 200 uri = /network-bootstore-config
17042024-09-06T23:42:13.079ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 402 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:50624 req_id = 22125957-6ef2-4e59-9134-1522fd4c91b6 response_code = 204 uri = /vpc-routes
17052024-09-06T23:42:13.080ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4014
17062024-09-06T23:42:13.084ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 183029ac-b909-4e34-9803-fdf7533f1a8f, kind: Service { id: 5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6 }, name: Name("external-dns-5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 179, 238])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7ef66645-e8c6-4c80-8f48-af73dfa3e97b, kind: Service { id: 195e4e82-1cc4-4f3b-a893-c1474bd26575 }, name: Name("external-dns-195e4e82-1cc4-4f3b-a893-c1474bd26575"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 212, 58])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 41a43463-6084-4d04-ad1b-f1d1ddccceaf, kind: Service { id: 9ad0f243-aeb1-455d-97ad-7dc9ca9c417c }, name: Name("nexus-9ad0f243-aeb1-455d-97ad-7dc9ca9c417c"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 248, 203])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a0534026-8f14-4794-ba39-35fb89d6d21c, kind: Service { id: cb51aad1-b6ab-44e1-8492-82c028d74c7f }, name: Name("nexus-cb51aad1-b6ab-44e1-8492-82c028d74c7f"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 170, 148])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ce58de05-7c50-42fb-bdcb-23c435be246c, kind: Service { id: 6c363cb4-229c-47a8-9d81-abd1714293f2 }, name: Name("nexus-6c363cb4-229c-47a8-9d81-abd1714293f2"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 218, 98])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
17072024-09-06T23:42:13.085ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
17082024-09-06T23:42:13.085ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17092024-09-06T23:42:13.085ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
17102024-09-06T23:42:13.085ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17112024-09-06T23:42:13.085ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
17122024-09-06T23:42:13.085ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17132024-09-06T23:42:13.085ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 5865 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:51808 req_id = 9552ca4f-2c80-46ff-8a33-e0adc93302a6 response_code = 200 uri = /v2p
17142024-09-06T23:42:13.085ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 5914 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:34232 req_id = 843dfdc6-5ea4-43f8-a72a-7f94690f1d54 response_code = 200 uri = /v2p
17152024-09-06T23:42:13.085ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 5969 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:61587 req_id = 0e9f9c07-3663-4560-950f-f7ea61483da8 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
17162024-09-06T23:42:13.088ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:50694
17172024-09-06T23:42:13.088ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 183029ac-b909-4e34-9803-fdf7533f1a8f, kind: Service { id: 5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6 }, name: Name("external-dns-5ed76b33-815e-4ee8-8f4d-39d8e83f1fb6"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 179, 238])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7ef66645-e8c6-4c80-8f48-af73dfa3e97b, kind: Service { id: 195e4e82-1cc4-4f3b-a893-c1474bd26575 }, name: Name("external-dns-195e4e82-1cc4-4f3b-a893-c1474bd26575"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 212, 58])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 41a43463-6084-4d04-ad1b-f1d1ddccceaf, kind: Service { id: 9ad0f243-aeb1-455d-97ad-7dc9ca9c417c }, name: Name("nexus-9ad0f243-aeb1-455d-97ad-7dc9ca9c417c"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 248, 203])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a0534026-8f14-4794-ba39-35fb89d6d21c, kind: Service { id: cb51aad1-b6ab-44e1-8492-82c028d74c7f }, name: Name("nexus-cb51aad1-b6ab-44e1-8492-82c028d74c7f"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 170, 148])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ce58de05-7c50-42fb-bdcb-23c435be246c, kind: Service { id: 6c363cb4-229c-47a8-9d81-abd1714293f2 }, name: Name("nexus-6c363cb4-229c-47a8-9d81-abd1714293f2"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 218, 98])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
17182024-09-06T23:42:13.088ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
17192024-09-06T23:42:13.088ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17202024-09-06T23:42:13.088ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
17212024-09-06T23:42:13.089ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17222024-09-06T23:42:13.089ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
17232024-09-06T23:42:13.089ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17242024-09-06T23:42:13.089ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 441 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:50694 req_id = 4743a6e5-fb2c-4396-91e1-9ff6d90d781f response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
17252024-09-06T23:42:13.117ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4038
17262024-09-06T23:42:13.167ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 87889 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:48315 req_id = 0d66a688-e48f-44e7-a507-98a2ef968ce9 response_code = 204 uri = /switch-ports
17272024-09-06T23:42:13.168ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 799 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:48315 req_id = 4915a0bb-cb69-4fb0-900b-6ac16d746d30 response_code = 200 uri = /network-bootstore-config
17282024-09-06T23:42:25.292ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 5414 local_addr = [fd00:1122:3344:101::1]:39558 method = GET remote_addr = [fd00:1122:3344:101::d]:50080 req_id = 48800355-3678-4a48-8289-b0b5aaa1b3bb response_code = 200 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c uri = /55267d89-b780-4916-86a8-64533b7bfd1c
17292024-09-06T23:42:25.479ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:50380
17302024-09-06T23:42:25.479ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 35 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:50380 req_id = 78a0c088-9d9c-494b-afa8-04feef9ee5ed response_code = 200 uri = /baseboard
17312024-09-06T23:42:42.458ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:55114
17322024-09-06T23:42:42.459ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 113 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:55114 req_id = ca133f69-d62d-4722-89d8-3d8bf6e58ce8 response_code = 200 uri = /vpc-routes
17332024-09-06T23:42:42.477ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 167 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:55114 req_id = 3f3e338a-46b2-4bb9-adcf-3a7fdc37fd02 response_code = 204 uri = /vpc-routes
17342024-09-06T23:42:42.572ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:45211
17352024-09-06T23:42:42.572ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 196 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:45211 req_id = 836027be-972a-4148-80d5-430b8c99c95d response_code = 200 uri = /v2p
17362024-09-06T23:42:42.634ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:39453
17372024-09-06T23:42:42.634ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 71 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:39453 req_id = 2497b3b1-5bce-46c9-99a8-215587775d98 response_code = 200 uri = /vpc-routes
17382024-09-06T23:42:42.642ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:60314
17392024-09-06T23:42:42.642ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4014
17402024-09-06T23:42:42.680ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4038
17412024-09-06T23:42:42.737ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 94759 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:60314 req_id = d49bc95f-85ed-4550-8b00-8ba4959bc664 response_code = 204 uri = /switch-ports
17422024-09-06T23:42:42.737ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 160 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:39453 req_id = 99e20492-f2ad-4b0d-80a2-ca921018baf6 response_code = 204 uri = /vpc-routes
17432024-09-06T23:42:42.737ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 116 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:60314 req_id = 6c5f54ec-4bcf-49ff-b6e2-0589f81647da response_code = 200 uri = /network-bootstore-config
17442024-09-06T23:42:42.743ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:54908
17452024-09-06T23:42:42.744ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 709 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:54908 req_id = ec8a9fc9-a16e-4e34-a044-a6044a9c38cd response_code = 200 uri = /v2p
17462024-09-06T23:42:42.745ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:49208
17472024-09-06T23:42:42.745ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 122 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:49208 req_id = 8eeb4277-ce4d-4bf0-9d5c-0795fe297d81 response_code = 200 uri = /v2p
17482024-09-06T23:42:42.749ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:58264
17492024-09-06T23:42:42.749ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4014
17502024-09-06T23:42:42.786ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4038
17512024-09-06T23:42:42.845ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 95636 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:58264 req_id = 74538ac1-2df9-48c2-b12a-12f11e874ae8 response_code = 204 uri = /switch-ports
17522024-09-06T23:42:42.845ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:46820
17532024-09-06T23:42:42.845ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:54469
17542024-09-06T23:42:42.845ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 92 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:46820 req_id = 2dd690b5-d44d-46e9-9423-a97b4cf2fcb7 response_code = 200 uri = /vpc-routes
17552024-09-06T23:42:42.846ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 238 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:58264 req_id = 94efe70c-702c-49b8-bbdf-ecc314565e31 response_code = 200 uri = /network-bootstore-config
17562024-09-06T23:42:42.846ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4014
17572024-09-06T23:42:42.855ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 183 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:46820 req_id = 8b58037c-8e42-4aa6-9bad-4c2884e41fdb response_code = 204 uri = /vpc-routes
17582024-09-06T23:42:42.888ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4038
17592024-09-06T23:42:42.945ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 99886 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:54469 req_id = 7c061e55-fcd8-48d5-b1d4-2b63a941e783 response_code = 204 uri = /switch-ports
17602024-09-06T23:42:42.946ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 138 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:54469 req_id = e80bb519-af97-422c-a512-78722e721af6 response_code = 200 uri = /network-bootstore-config
17612024-09-06T23:42:55.375ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 5096 local_addr = [fd00:1122:3344:101::1]:39558 method = GET remote_addr = [fd00:1122:3344:101::d]:50080 req_id = dd68a9e1-d20a-4ca6-9db4-27ee0017967e response_code = 200 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c uri = /55267d89-b780-4916-86a8-64533b7bfd1c
17622024-09-06T23:42:55.512ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:47828
17632024-09-06T23:42:55.512ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 611 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:47828 req_id = 7290bdb8-e77f-493b-b570-bf716631faa6 response_code = 200 uri = /baseboard
17642024-09-06T23:43:12.531ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:59945
17652024-09-06T23:43:12.531ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 128 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:59945 req_id = 08d28354-5481-4a45-9f28-187c747c848a response_code = 200 uri = /vpc-routes
17662024-09-06T23:43:12.565ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 2717 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:59945 req_id = 4d941df9-82ae-4600-9e41-50f6cf84e062 response_code = 204 uri = /vpc-routes
17672024-09-06T23:43:12.763ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:44309
17682024-09-06T23:43:12.764ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1174 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:44309 req_id = 923ee934-34d5-4dd6-957e-9388aa50cb42 response_code = 200 uri = /v2p
17692024-09-06T23:43:12.797ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:63496
17702024-09-06T23:43:12.797ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 74 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:63496 req_id = d424b328-ec38-4357-95f2-eba71510d618 response_code = 200 uri = /vpc-routes
17712024-09-06T23:43:12.810ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 807 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:63496 req_id = 8e656743-977a-47d2-88cb-29dfab4d502f response_code = 204 uri = /vpc-routes
17722024-09-06T23:43:12.834ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:49924
17732024-09-06T23:43:12.834ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4014
17742024-09-06T23:43:12.884ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4038
17752024-09-06T23:43:12.943ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 108692 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:49924 req_id = 3623c17e-8b7f-42c8-b3bc-6257f85a91ae response_code = 204 uri = /switch-ports
17762024-09-06T23:43:12.943ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:56040
17772024-09-06T23:43:12.943ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:43738
17782024-09-06T23:43:12.943ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 215 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:56040 req_id = ce94653d-0c4a-489c-be17-d0d80812800b response_code = 200 uri = /v2p
17792024-09-06T23:43:12.943ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 237 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:49924 req_id = e9fd2517-e1fd-4561-9098-27d33bdfa95f response_code = 200 uri = /network-bootstore-config
17802024-09-06T23:43:12.943ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4014
17812024-09-06T23:43:12.948ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:44570
17822024-09-06T23:43:12.949ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1216 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:44570 req_id = b7e72eb8-2a17-4247-8708-9858ff23f181 response_code = 200 uri = /vpc-routes
17832024-09-06T23:43:12.973ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 236 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:44570 req_id = cf9cbcb7-6eb8-4aef-a6ad-4bd8f37a459f response_code = 204 uri = /vpc-routes
17842024-09-06T23:43:12.992ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4038
17852024-09-06T23:43:13.067ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 123454 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:43738 req_id = 45a1a7d7-c0d6-47ec-87f3-fc7bae139f4f response_code = 204 uri = /switch-ports
17862024-09-06T23:43:13.067ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 172 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:43738 req_id = b99911c5-de7a-4eb7-a361-7313590714c4 response_code = 200 uri = /network-bootstore-config
17872024-09-06T23:43:13.078ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:35197
17882024-09-06T23:43:13.078ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4014
17892024-09-06T23:43:13.112ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4038
17902024-09-06T23:43:13.162ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 83852 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:35197 req_id = d54d05d2-ea3c-4e37-ab69-076b75806915 response_code = 204 uri = /switch-ports
17912024-09-06T23:43:13.163ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:63350
17922024-09-06T23:43:13.163ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 221 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:35197 req_id = 72f2dacd-569b-442a-8714-76e8e6570124 response_code = 200 uri = /network-bootstore-config
17932024-09-06T23:43:13.163ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 225 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:63350 req_id = 41056343-6e1d-41f8-8d99-6bfc253bf0b1 response_code = 200 uri = /v2p
17942024-09-06T23:43:25.376ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 6449 local_addr = [fd00:1122:3344:101::1]:39558 method = GET remote_addr = [fd00:1122:3344:101::d]:50080 req_id = 99d34e89-0e15-44cb-aa77-46e1ce4122f0 response_code = 200 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c uri = /55267d89-b780-4916-86a8-64533b7bfd1c
17952024-09-06T23:43:25.550ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:39995
17962024-09-06T23:43:25.551ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 110 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:39995 req_id = 13e6e23a-7cd1-4670-914f-c5a930483174 response_code = 200 uri = /baseboard
17972024-09-06T23:43:42.365ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:60329
17982024-09-06T23:43:42.366ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 838 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:60329 req_id = 2a665742-230d-4a68-af2e-6785e2a3090b response_code = 200 uri = /vpc-routes
17992024-09-06T23:43:42.388ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 147 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:60329 req_id = b59718d6-c7a9-490d-ab7f-b98d8b2e4e92 response_code = 204 uri = /vpc-routes
18002024-09-06T23:43:42.575ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:51899
18012024-09-06T23:43:42.575ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 81 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:51899 req_id = cd6e9f75-c2f1-4953-bc24-15808fae3d68 response_code = 200 uri = /vpc-routes
18022024-09-06T23:43:42.589ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:61891
18032024-09-06T23:43:42.589ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 187 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:61891 req_id = 6d1cbf07-a7a5-41ec-8209-34309e2be912 response_code = 200 uri = /v2p
18042024-09-06T23:43:42.595ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 104 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:51899 req_id = 07a02793-2546-445f-81f9-629affe2b36b response_code = 204 uri = /vpc-routes
18052024-09-06T23:43:42.655ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:56856
18062024-09-06T23:43:42.655ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4014
18072024-09-06T23:43:42.698ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4038
18082024-09-06T23:43:42.766ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 111238 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:56856 req_id = e7513d12-0471-4ef9-9fee-3c1c990d1c31 response_code = 204 uri = /switch-ports
18092024-09-06T23:43:42.766ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:58688
18102024-09-06T23:43:42.766ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 90 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:58688 req_id = 4b91f3eb-0545-43b5-a4b9-7ae163d95b63 response_code = 200 uri = /vpc-routes
18112024-09-06T23:43:42.768ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1139 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:56856 req_id = e92af7b1-b944-4cdc-9335-4432729de413 response_code = 200 uri = /network-bootstore-config
18122024-09-06T23:43:42.773ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:34531
18132024-09-06T23:43:42.773ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 164 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:34531 req_id = fda97be0-51bc-4f71-af00-4e78974eb6b9 response_code = 200 uri = /v2p
18142024-09-06T23:43:42.778ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 143 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:58688 req_id = 69cd497c-44b8-40e2-a3ac-05eca4eb0395 response_code = 204 uri = /vpc-routes
18152024-09-06T23:43:42.798ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:52001
18162024-09-06T23:43:42.798ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4014
18172024-09-06T23:43:42.835ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4038
18182024-09-06T23:43:42.892ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 93574 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:52001 req_id = e99098ec-125d-4ffc-9d9c-22192de3963c response_code = 204 uri = /switch-ports
18192024-09-06T23:43:42.892ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:35512
18202024-09-06T23:43:42.892ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:42795
18212024-09-06T23:43:42.892ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 279 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:52001 req_id = 2b3e585d-2d16-4138-8b0c-578bf3f52891 response_code = 200 uri = /network-bootstore-config
18222024-09-06T23:43:42.892ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4014
18232024-09-06T23:43:42.892ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 236 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:42795 req_id = e40a30f2-90b2-4c40-b716-739f8241f56d response_code = 200 uri = /v2p
18242024-09-06T23:43:42.933ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4038
18252024-09-06T23:43:42.991ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 99087 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:35512 req_id = 1443203f-1b3a-4500-b1ff-5657a10e22fe response_code = 204 uri = /switch-ports
18262024-09-06T23:43:42.992ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 164 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:35512 req_id = b4892e52-7263-4477-8c24-f645f16aab28 response_code = 200 uri = /network-bootstore-config
18272024-09-06T23:43:46.743ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:40386
18282024-09-06T23:43:46.743ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 208 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:40386 req_id = 5630eeff-078c-44d7-aa62-1cd61a5831b9 response_code = 200 uri = /v2p
18292024-09-06T23:43:46.743ZINFOSledAgent (PortManager): Mapping virtual NIC to physical host file = illumos-utils/src/opte/port_manager.rs:778 mapping = VirtualNetworkInterfaceHost { virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 245, 183, 82])), physical_host_ip: fd00:1122:3344:101::1, vni: Vni(12801096) }
18302024-09-06T23:43:46.743ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 100 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:40386 req_id = 24a48909-54b5-43a4-9ae6-f6bfe658fc89 response_code = 204 uri = /v2p
18312024-09-06T23:43:46.908ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:35952
18322024-09-06T23:43:46.908ZINFOSledAgent (InstanceManager): ensuring instance is registered file = sled-agent/src/instance_manager.rs:570 hardware = InstanceHardware { properties: InstanceProperties { ncpus: InstanceCpuCount(2), memory: ByteCount(1073741824), hostname: Hostname("localshark") }, nics: [NetworkInterface { id: b5c28a64-1c4e-46ba-ad11-05e3da186388, kind: Instance { id: 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 245, 183, 82])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(12801096), primary: true, slot: 0, transit_ips: [] }], source_nat: SourceNatConfig { ip: 10.151.2.175, first_port: 0, last_port: 16383 }, ephemeral_ip: Some(10.151.2.176), floating_ips: [], firewall_rules: [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: b5c28a64-1c4e-46ba-ad11-05e3da186388, kind: Instance { id: 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 245, 183, 82])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(12801096), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: b5c28a64-1c4e-46ba-ad11-05e3da186388, kind: Instance { id: 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 245, 183, 82])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(12801096), primary: true, slot: 0, transit_ips: [] }], filter_hosts: Some([Vpc(Vni(12801096))]), filter_ports: None, filter_protocols: None, action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: b5c28a64-1c4e-46ba-ad11-05e3da186388, kind: Instance { id: 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 245, 183, 82])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(12801096), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(22), last: L4Port(22) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }], dhcp_config: DhcpConfig { dns_servers: [1.1.1.1, 9.9.9.9], host_domain: None, search_domains: [] }, disks: [DiskRequest { device: "nvme", name: "disk-d4899155ad2", read_only: false, slot: Slot(0), volume_construction_request: Volume { block_size: 512, id: e36f5160-089e-4ecf-9226-7749e7726e24, read_only_parent: Some(Volume { block_size: 512, id: 69e49366-bce7-4e8f-90e1-0efe5a3efe76, read_only_parent: None, sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 2, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: da0e59cc-ac4d-4d90-96e5-71b0e55ef0eb, key: Some("nu1EosAauKBPFS9rzRVhkVFdl/0PMZyH+ueLsGdraRk="), key_pem: None, lossy: false, read_only: true, root_cert_pem: None, target: ["[fd00:1122:3344:101::16]:19001", "[fd00:1122:3344:101::1a]:19002", "[fd00:1122:3344:101::15]:19001"] } }] }), sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 1, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: e36f5160-089e-4ecf-9226-7749e7726e24, key: Some("a2uXzwnIDX7/mZuwnvLsC4p2sraNRq7MoZaeqSt4zHA="), key_pem: None, lossy: false, read_only: false, root_cert_pem: None, target: ["[fd00:1122:3344:101::14]:19001", "[fd00:1122:3344:101::15]:19002", "[fd00:1122:3344:101::17]:19000"] } }] } }], cloud_init_bytes: Some(..) } instance_id = 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 instance_runtime = InstanceRuntimeState { propolis_id: Some(d4ea02d9-e88d-4758-8341-98e3f57f2467 (propolis)), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2024-09-06T23:43:46.448627190Z } metadata = InstanceMetadata { silo_id: 17ec7f0a-a8ca-4d33-aaeb-e6c5e850a409, project_id: 1fe9032f-3c93-43f8-9cd9-101631e85beb } propolis_addr = [fd00:1122:3344:101::1:0]:12400 propolis_id = d4ea02d9-e88d-4758-8341-98e3f57f2467 vmm_runtime = VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2024-09-06T23:43:46.905460644Z }
18332024-09-06T23:43:46.908ZINFOSledAgent (InstanceManager): registering new instance file = sled-agent/src/instance_manager.rs:603 instance_id = 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 propolis_id = d4ea02d9-e88d-4758-8341-98e3f57f2467
18342024-09-06T23:43:46.908ZINFOSledAgent (InstanceManager): initializing new Instance file = sled-agent/src/instance.rs:1008 instance_id = 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 migration_id = None propolis_id = d4ea02d9-e88d-4758-8341-98e3f57f2467 state = InstanceInitialState { hardware: InstanceHardware { properties: InstanceProperties { ncpus: InstanceCpuCount(2), memory: ByteCount(1073741824), hostname: Hostname("localshark") }, nics: [NetworkInterface { id: b5c28a64-1c4e-46ba-ad11-05e3da186388, kind: Instance { id: 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 245, 183, 82])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(12801096), primary: true, slot: 0, transit_ips: [] }], source_nat: SourceNatConfig { ip: 10.151.2.175, first_port: 0, last_port: 16383 }, ephemeral_ip: Some(10.151.2.176), floating_ips: [], firewall_rules: [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: b5c28a64-1c4e-46ba-ad11-05e3da186388, kind: Instance { id: 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 245, 183, 82])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(12801096), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: b5c28a64-1c4e-46ba-ad11-05e3da186388, kind: Instance { id: 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 245, 183, 82])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(12801096), primary: true, slot: 0, transit_ips: [] }], filter_hosts: Some([Vpc(Vni(12801096))]), filter_ports: None, filter_protocols: None, action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: b5c28a64-1c4e-46ba-ad11-05e3da186388, kind: Instance { id: 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 245, 183, 82])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(12801096), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(22), last: L4Port(22) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }], dhcp_config: DhcpConfig { dns_servers: [1.1.1.1, 9.9.9.9], host_domain: None, search_domains: [] }, disks: [DiskRequest { device: "nvme", name: "disk-d4899155ad2", read_only: false, slot: Slot(0), volume_construction_request: Volume { block_size: 512, id: e36f5160-089e-4ecf-9226-7749e7726e24, read_only_parent: Some(Volume { block_size: 512, id: 69e49366-bce7-4e8f-90e1-0efe5a3efe76, read_only_parent: None, sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 2, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: da0e59cc-ac4d-4d90-96e5-71b0e55ef0eb, key: Some("nu1EosAauKBPFS9rzRVhkVFdl/0PMZyH+ueLsGdraRk="), key_pem: None, lossy: false, read_only: true, root_cert_pem: None, target: ["[fd00:1122:3344:101::16]:19001", "[fd00:1122:3344:101::1a]:19002", "[fd00:1122:3344:101::15]:19001"] } }] }), sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 1, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: e36f5160-089e-4ecf-9226-7749e7726e24, key: Some("a2uXzwnIDX7/mZuwnvLsC4p2sraNRq7MoZaeqSt4zHA="), key_pem: None, lossy: false, read_only: false, root_cert_pem: None, target: ["[fd00:1122:3344:101::14]:19001", "[fd00:1122:3344:101::15]:19002", "[fd00:1122:3344:101::17]:19000"] } }] } }], cloud_init_bytes: Some(..) }, vmm_runtime: VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2024-09-06T23:43:46.905460644Z }, propolis_addr: [fd00:1122:3344:101::1:0]:12400, migration_id: None }
18352024-09-06T23:43:46.909ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1014 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:35952 req_id = fb2ba5a3-bcfb-4562-afdf-0ff70eee0344 response_code = 200 uri = /vmms/d4ea02d9-e88d-4758-8341-98e3f57f2467
18362024-09-06T23:43:47.018ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:61436
18372024-09-06T23:43:47.019ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 587 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:61436 req_id = afa1bd6a-4dd7-4b00-bc6e-09220d2ee313 response_code = 200 uri = /vmms/d4ea02d9-e88d-4758-8341-98e3f57f2467/state
18382024-09-06T23:43:47.019ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte6", ip: 172.30.0.5, mac: MacAddr6([168, 64, 37, 245, 183, 82]), slot: 0, vni: Vni { inner: 12801096 }, subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.0.1 } }) }
18392024-09-06T23:43:47.019ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
18402024-09-06T23:43:47.061ZINFOSledAgent (InstanceManager): Configuring new Omicron zone: oxz_propolis-server_d4ea02d9-e88d-4758-8341-98e3f57f2467 file = illumos-utils/src/zone.rs:316 instance_id = 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 propolis_id = d4ea02d9-e88d-4758-8341-98e3f57f2467
18412024-09-06T23:43:47.082ZINFOSledAgent (InstanceManager): Installing Omicron zone: oxz_propolis-server_d4ea02d9-e88d-4758-8341-98e3f57f2467 file = illumos-utils/src/zone.rs:354 instance_id = 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 propolis_id = d4ea02d9-e88d-4758-8341-98e3f57f2467
18422024-09-06T23:43:48.035ZINFOSledAgent (InstanceManager): Profile for oxz_propolis-server_d4ea02d9-e88d-4758-8341-98e3f57f2467: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="system/illumos/propolis-server"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlInstance0'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::1:0'/> <propval type="astring" name="listen_port" value='12400'/> <propval type="astring" name="metric_addr" value='dns'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34 instance_id = 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 propolis_id = d4ea02d9-e88d-4758-8341-98e3f57f2467
18432024-09-06T23:43:48.036ZINFOSledAgent (InstanceManager): Booting oxz_propolis-server_d4ea02d9-e88d-4758-8341-98e3f57f2467 zone file = illumos-utils/src/running_zone.rs:488 instance_id = 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 propolis_id = d4ea02d9-e88d-4758-8341-98e3f57f2467 zone = oxz_propolis-server_d4ea02d9-e88d-4758-8341-98e3f57f2467
18442024-09-06T23:43:48.208ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_d4ea02d9-e88d-4758-8341-98e3f57f2467") failed: Property not found. retry in 59.907488ms file = illumos-utils/src/svc.rs:36 instance_id = 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 propolis_id = d4ea02d9-e88d-4758-8341-98e3f57f2467 zone = oxz_propolis-server_d4ea02d9-e88d-4758-8341-98e3f57f2467
18452024-09-06T23:43:48.293ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_d4ea02d9-e88d-4758-8341-98e3f57f2467") failed: Property not found. retry in 77.373354ms file = illumos-utils/src/svc.rs:36 instance_id = 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 propolis_id = d4ea02d9-e88d-4758-8341-98e3f57f2467 zone = oxz_propolis-server_d4ea02d9-e88d-4758-8341-98e3f57f2467
18462024-09-06T23:43:48.385ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_d4ea02d9-e88d-4758-8341-98e3f57f2467") failed: Property not found. retry in 130.82845ms file = illumos-utils/src/svc.rs:36 instance_id = 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 propolis_id = d4ea02d9-e88d-4758-8341-98e3f57f2467 zone = oxz_propolis-server_d4ea02d9-e88d-4758-8341-98e3f57f2467
18472024-09-06T23:43:48.532ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_d4ea02d9-e88d-4758-8341-98e3f57f2467") failed: Property not found. retry in 414.710187ms file = illumos-utils/src/svc.rs:36 instance_id = 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 propolis_id = d4ea02d9-e88d-4758-8341-98e3f57f2467 zone = oxz_propolis-server_d4ea02d9-e88d-4758-8341-98e3f57f2467
18482024-09-06T23:43:48.962ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_d4ea02d9-e88d-4758-8341-98e3f57f2467") failed: Property not found. retry in 740.133142ms file = illumos-utils/src/svc.rs:36 instance_id = 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 propolis_id = d4ea02d9-e88d-4758-8341-98e3f57f2467 zone = oxz_propolis-server_d4ea02d9-e88d-4758-8341-98e3f57f2467
18492024-09-06T23:43:49.714ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_d4ea02d9-e88d-4758-8341-98e3f57f2467") failed: Property not found. retry in 1.144206525s file = illumos-utils/src/svc.rs:36 instance_id = 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 propolis_id = d4ea02d9-e88d-4758-8341-98e3f57f2467 zone = oxz_propolis-server_d4ea02d9-e88d-4758-8341-98e3f57f2467
18502024-09-06T23:43:50.871ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_d4ea02d9-e88d-4758-8341-98e3f57f2467") failed: Property not found. retry in 878.453783ms file = illumos-utils/src/svc.rs:36 instance_id = 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 propolis_id = d4ea02d9-e88d-4758-8341-98e3f57f2467 zone = oxz_propolis-server_d4ea02d9-e88d-4758-8341-98e3f57f2467
18512024-09-06T23:43:51.760ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_d4ea02d9-e88d-4758-8341-98e3f57f2467") failed: Property not found. retry in 1.078172218s file = illumos-utils/src/svc.rs:36 instance_id = 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 propolis_id = d4ea02d9-e88d-4758-8341-98e3f57f2467 zone = oxz_propolis-server_d4ea02d9-e88d-4758-8341-98e3f57f2467
18522024-09-06T23:43:52.850ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_d4ea02d9-e88d-4758-8341-98e3f57f2467") failed: Property not found. retry in 1.023952907s file = illumos-utils/src/svc.rs:36 instance_id = 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 propolis_id = d4ea02d9-e88d-4758-8341-98e3f57f2467 zone = oxz_propolis-server_d4ea02d9-e88d-4758-8341-98e3f57f2467
18532024-09-06T23:43:53.903ZINFOSledAgent (InstanceManager): Started propolis in zone: oxz_propolis-server_d4ea02d9-e88d-4758-8341-98e3f57f2467 file = sled-agent/src/instance.rs:1466 instance_id = 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 propolis_id = d4ea02d9-e88d-4758-8341-98e3f57f2467
18542024-09-06T23:43:53.912ZINFOSledAgent (InstanceManager): Propolis SMF service is online file = sled-agent/src/instance.rs:1475 instance_id = 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 propolis_id = d4ea02d9-e88d-4758-8341-98e3f57f2467
18552024-09-06T23:43:53.948ZINFOSledAgent (InstanceManager): Propolis HTTP server online file = sled-agent/src/instance.rs:1499 instance_id = 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 propolis_id = d4ea02d9-e88d-4758-8341-98e3f57f2467
18562024-09-06T23:43:54.090ZINFOSledAgent (InstanceManager): result of instance_ensure call is Ok(InstanceEnsureResponse { migrate: None }) file = sled-agent/src/instance.rs:744 instance_id = 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 propolis_id = d4ea02d9-e88d-4758-8341-98e3f57f2467
18572024-09-06T23:43:54.091ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Creating), migration_in: None, migration_out: None, time: 2024-09-06T23:43:54.091598629Z } file = sled-agent/src/instance.rs:594 instance_id = 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 propolis_id = d4ea02d9-e88d-4758-8341-98e3f57f2467
18582024-09-06T23:43:54.091ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:616 instance_id = 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 new_vmm_state = VmmRuntimeState { state: Starting, gen: Generation(3), time_updated: 2024-09-06T23:43:54.091598629Z } propolis_id = d4ea02d9-e88d-4758-8341-98e3f57f2467
18592024-09-06T23:43:54.091ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:514 instance_id = 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 propolis_id = d4ea02d9-e88d-4758-8341-98e3f57f2467 state = SledVmmState { vmm_state: VmmRuntimeState { state: Starting, gen: Generation(3), time_updated: 2024-09-06T23:43:54.091598629Z }, migration_in: None, migration_out: None }
18602024-09-06T23:43:54.141ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Running), migration_in: None, migration_out: None, time: 2024-09-06T23:43:54.141589837Z } file = sled-agent/src/instance.rs:594 instance_id = 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 propolis_id = d4ea02d9-e88d-4758-8341-98e3f57f2467
18612024-09-06T23:43:54.141ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:616 instance_id = 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 new_vmm_state = VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2024-09-06T23:43:54.141589837Z } propolis_id = d4ea02d9-e88d-4758-8341-98e3f57f2467
18622024-09-06T23:43:54.141ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:514 instance_id = 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 propolis_id = d4ea02d9-e88d-4758-8341-98e3f57f2467 state = SledVmmState { vmm_state: VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2024-09-06T23:43:54.141589837Z }, migration_in: None, migration_out: None }
18632024-09-06T23:43:55.376ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 6068 local_addr = [fd00:1122:3344:101::1]:39558 method = GET remote_addr = [fd00:1122:3344:101::d]:50080 req_id = aed9f0f1-7c0d-4257-aa0c-cbb493e10b52 response_code = 200 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c uri = /55267d89-b780-4916-86a8-64533b7bfd1c
18642024-09-06T23:43:55.588ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:42210
18652024-09-06T23:43:55.588ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 38 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:42210 req_id = 44e2b5ef-0ca6-4845-9591-6cb43e5b0423 response_code = 200 uri = /baseboard
18662024-09-06T23:44:12.528ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:39550
18672024-09-06T23:44:12.530ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1868 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:39550 req_id = ec5229e9-670f-4bd0-81d9-56f8de8bb3ce response_code = 200 uri = /vmms/d4ea02d9-e88d-4758-8341-98e3f57f2467/state
18682024-09-06T23:44:12.745ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:65412
18692024-09-06T23:44:12.745ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 61 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:65412 req_id = 10dfac14-40fa-4ff8-af32-e7315e1a78b1 response_code = 200 uri = /vpc-routes
18702024-09-06T23:44:12.766ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:52146
18712024-09-06T23:44:12.766ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 150 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:52146 req_id = c89f5aac-8745-4eb7-8427-e0c122c42887 response_code = 200 uri = /v2p
18722024-09-06T23:44:12.815ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 229 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:65412 req_id = 17ec3dfb-8365-400c-8a2e-7e4bd0a2ae71 response_code = 204 uri = /vpc-routes
18732024-09-06T23:44:12.847ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:41597
18742024-09-06T23:44:12.847ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 42 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:41597 req_id = da6bfa2f-9b37-4bee-b731-cc240c498172 response_code = 200 uri = /vpc-routes
18752024-09-06T23:44:12.849ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:58292
18762024-09-06T23:44:12.850ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4014
18772024-09-06T23:44:12.894ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4038
18782024-09-06T23:44:12.948ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 98670 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:58292 req_id = a5fb446a-27c8-4fe5-829b-3abf98dc5aba response_code = 204 uri = /switch-ports
18792024-09-06T23:44:12.948ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:36549
18802024-09-06T23:44:12.949ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1006 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:36549 req_id = 0957ce50-54ff-41bc-9038-0cfe34af6b9e response_code = 200 uri = /vpc-routes
18812024-09-06T23:44:12.949ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1062 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:58292 req_id = 32a9d2f8-e228-49fc-84e4-666e056817f8 response_code = 200 uri = /network-bootstore-config
18822024-09-06T23:44:12.996ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:59994
18832024-09-06T23:44:12.996ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 158 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:59994 req_id = 804a147b-f044-4789-b299-db3627c1c97d response_code = 200 uri = /vmms/d4ea02d9-e88d-4758-8341-98e3f57f2467/state
18842024-09-06T23:44:13.007ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:54528
18852024-09-06T23:44:13.008ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 930 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:54528 req_id = 9044330a-1a61-4ca4-ad75-c2260cd8ed7c response_code = 200 uri = /vmms/d4ea02d9-e88d-4758-8341-98e3f57f2467/state
18862024-09-06T23:44:13.015ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 137 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:36549 req_id = 735fa89a-35e5-4839-899e-6829e3391a3a response_code = 204 uri = /vpc-routes
18872024-09-06T23:44:13.027ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 86 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:41597 req_id = 8456efe4-f88c-4aab-95bc-bcd9c266355d response_code = 204 uri = /vpc-routes
18882024-09-06T23:44:13.130ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:57546
18892024-09-06T23:44:13.131ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 125 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:57546 req_id = ef67b5f8-c64f-4552-8e8c-e73420d9f253 response_code = 200 uri = /v2p
18902024-09-06T23:44:13.143ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:60606
18912024-09-06T23:44:13.144ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 809 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:60606 req_id = 4ce60fdd-906b-44a3-95f8-31d303884b6b response_code = 200 uri = /v2p
18922024-09-06T23:44:13.156ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:35946
18932024-09-06T23:44:13.156ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4014
18942024-09-06T23:44:13.195ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4038
18952024-09-06T23:44:13.249ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 93027 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:35946 req_id = a0bf3144-1ece-45a4-972c-ede8138ec033 response_code = 204 uri = /switch-ports
18962024-09-06T23:44:13.249ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:38088
18972024-09-06T23:44:13.249ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 262 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:35946 req_id = f01bb473-fbb4-4286-97d0-aa636b676035 response_code = 200 uri = /network-bootstore-config
18982024-09-06T23:44:13.249ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4014
18992024-09-06T23:44:13.287ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4038
19002024-09-06T23:44:13.342ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 93022 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:38088 req_id = 19e0d776-db55-425a-94ac-b7dea35b0807 response_code = 204 uri = /switch-ports
19012024-09-06T23:44:13.343ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 906 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:38088 req_id = 3db7c3aa-bc03-4687-b330-6b56daa9d9e0 response_code = 200 uri = /network-bootstore-config
19022024-09-06T23:44:20.449ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:54920
19032024-09-06T23:44:20.450ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1061 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:54920 req_id = 610e9e0b-31e7-4192-b4cc-39b918830ff4 response_code = 200 uri = /vmms/d4ea02d9-e88d-4758-8341-98e3f57f2467/state
19042024-09-06T23:44:20.451ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Stopping), migration_in: None, migration_out: None, time: 2024-09-06T23:44:20.451015421Z } file = sled-agent/src/instance.rs:594 instance_id = 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 propolis_id = d4ea02d9-e88d-4758-8341-98e3f57f2467
19052024-09-06T23:44:20.451ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:616 instance_id = 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 new_vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2024-09-06T23:44:20.451015421Z } propolis_id = d4ea02d9-e88d-4758-8341-98e3f57f2467
19062024-09-06T23:44:20.451ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:514 instance_id = 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 propolis_id = d4ea02d9-e88d-4758-8341-98e3f57f2467 state = SledVmmState { vmm_state: VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2024-09-06T23:44:20.451015421Z }, migration_in: None, migration_out: None }
19072024-09-06T23:44:20.469ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Destroyed), migration_in: None, migration_out: None, time: 2024-09-06T23:44:20.469899346Z } file = sled-agent/src/instance.rs:594 instance_id = 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 propolis_id = d4ea02d9-e88d-4758-8341-98e3f57f2467
19082024-09-06T23:44:20.469ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:616 instance_id = 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 new_vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(7), time_updated: 2024-09-06T23:44:20.469899346Z } propolis_id = d4ea02d9-e88d-4758-8341-98e3f57f2467
19092024-09-06T23:44:20.469ZINFOSledAgent (InstanceManager): terminating VMM that has exited file = sled-agent/src/instance.rs:630 instance_id = 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 propolis_id = d4ea02d9-e88d-4758-8341-98e3f57f2467
19102024-09-06T23:44:20.469ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
19112024-09-06T23:44:20.470ZWARNSledAgent (kstat-sampler-worker): target removed with queued samples file = oximeter/instruments/src/kstat/sampler.rs:557 id = 5216507778438846536 n_samples = 12 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c
19122024-09-06T23:44:20.470ZWARNSledAgent (kstat-sampler-worker): target removed with queued samples file = oximeter/instruments/src/kstat/sampler.rs:557 id = 11801522055854723338 n_samples = 12 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c
19132024-09-06T23:44:20.471ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
19142024-09-06T23:44:20.471ZINFOSledAgent (ZoneBundler): creating zone bundle context = ZoneBundleContext { storage_dirs: ["/pool/int/358fc814-3f2a-4b1a-aebb-894434d72d01/debug/bundle/zone", "/pool/int/ca24db04-f37b-40b1-83c2-23476bec5a4a/debug/bundle/zone"], cause: TerminatedInstance, extra_log_dirs: ["/pool/ext/652da97e-d553-4457-bc42-d10c0c455967/crypt/debug", "/pool/ext/1bae97fb-c4f0-4d17-b41c-409574fb9e6e/crypt/debug", "/pool/ext/eae5bedf-a363-4cf3-b83a-efcb44445301/crypt/debug", "/pool/ext/ab9a060c-a3d5-48de-ba57-beaa10d18520/crypt/debug", "/pool/ext/fd8d5a1e-9723-41c1-aa2b-460a6959d6c7/crypt/debug", "/pool/ext/b79af253-dd33-4fe7-98c1-790c50e27394/crypt/debug", "/pool/ext/cc3f0ccd-963c-4d0c-843d-e8810b230566/crypt/debug", "/pool/ext/f8e03f3f-0403-4c51-926a-d5ce11d8408e/crypt/debug", "/pool/ext/907237be-faec-4233-8c3e-1918ec810764/crypt/debug"] } file = sled-agent/src/zone_bundle.rs:339 zone_name = oxz_propolis-server_d4ea02d9-e88d-4758-8341-98e3f57f2467
19152024-09-06T23:44:21.092ZINFOSledAgent (ZoneBundler): finished zone bundle file = sled-agent/src/zone_bundle.rs:1142 metadata = ZoneBundleMetadata { id: ZoneBundleId { zone_name: "oxz_propolis-server_d4ea02d9-e88d-4758-8341-98e3f57f2467", bundle_id: 019e0151-4a47-4ebd-a2a1-1fa9e35bbc3f }, time_created: 2024-09-06T23:44:20.471893833Z, version: 0, cause: TerminatedInstance }
19162024-09-06T23:44:21.092ZWARNSledAgent (InstanceManager): Halting and removing zone: oxz_propolis-server_d4ea02d9-e88d-4758-8341-98e3f57f2467 file = sled-agent/src/instance.rs:843 instance_id = 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 propolis_id = d4ea02d9-e88d-4758-8341-98e3f57f2467
19172024-09-06T23:44:22.867ZINFOSledAgent (InstanceManager): halt_and_remove_logged: Previous zone state: Running file = illumos-utils/src/zone.rs:270 instance_id = 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 propolis_id = d4ea02d9-e88d-4758-8341-98e3f57f2467
19182024-09-06T23:44:22.883ZINFOSledAgent (PortManager): Removed route set for subnet file = illumos-utils/src/opte/port_manager.rs:904 id = RouterId { vni: Vni(12801096), kind: System }
19192024-09-06T23:44:22.883ZINFOSledAgent (PortManager): Removed route set for subnet file = illumos-utils/src/opte/port_manager.rs:904 id = RouterId { vni: Vni(12801096), kind: Custom(V4(Ipv4Net { addr: 172.30.0.0, width: 22 })) }
19202024-09-06T23:44:22.884ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:514 instance_id = 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 propolis_id = d4ea02d9-e88d-4758-8341-98e3f57f2467 state = SledVmmState { vmm_state: VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2024-09-06T23:44:22.884024818Z }, migration_in: None, migration_out: None }
19212024-09-06T23:44:22.939ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:514 instance_id = 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 propolis_id = d4ea02d9-e88d-4758-8341-98e3f57f2467 state = SledVmmState { vmm_state: VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2024-09-06T23:44:22.884024818Z }, migration_in: None, migration_out: None }
19222024-09-06T23:44:22.943ZINFOSledAgent (InstanceManager): State monitoring task complete file = sled-agent/src/instance.rs:780 instance_id = 747595ba-6ecd-44ef-be5f-dcf9a640cfb1 propolis_id = d4ea02d9-e88d-4758-8341-98e3f57f2467
19232024-09-06T23:44:23.414ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:54074
19242024-09-06T23:44:23.414ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 45 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:54074 req_id = 6deee36c-b28d-4ed4-bf19-36be96740c52 response_code = 200 uri = /vpc-routes
19252024-09-06T23:44:23.428ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 105 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:54074 req_id = 526784c4-6532-4d03-a698-1cf4d71a1dc1 response_code = 204 uri = /vpc-routes
19262024-09-06T23:44:23.497ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:53187
19272024-09-06T23:44:23.497ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 128 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:53187 req_id = ef7cceb0-c3b5-41a3-8138-33e58707d098 response_code = 200 uri = /v2p
19282024-09-06T23:44:24.735ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:49816
19292024-09-06T23:44:24.735ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 133 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:49816 req_id = c4a70212-b1d1-4ff8-a9e1-23bb3727f184 response_code = 200 uri = /v2p
19302024-09-06T23:44:25.377ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 5828 local_addr = [fd00:1122:3344:101::1]:39558 method = GET remote_addr = [fd00:1122:3344:101::d]:50080 req_id = 1149a315-daef-4ca9-b933-4b49e1ae70d7 response_code = 200 sled_id = 55267d89-b780-4916-86a8-64533b7bfd1c uri = /55267d89-b780-4916-86a8-64533b7bfd1c
19312024-09-06T23:44:25.621ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:34593
19322024-09-06T23:44:25.621ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 36 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:34593 req_id = 54df4f1c-aed8-4755-a8c7-ee7e43e36087 response_code = 200 uri = /baseboard
19332024-09-06T23:44:25.837ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:34914
19342024-09-06T23:44:25.837ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 47 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:34914 req_id = f7ea5b90-0812-41b2-97a2-78e6581c67a9 response_code = 200 uri = /vpc-routes
19352024-09-06T23:44:25.846ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 656 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:34914 req_id = 39614fac-d437-4c85-9e25-441e546c9d92 response_code = 204 uri = /vpc-routes