Max level shown:
1 [ Aug 28 02:08:49 Enabled. ]
2 [ Aug 28 02:08:49 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/sled-agent/sled-agent run /opt/oxide/sled-agent/pkg/config.toml &"). ]
3 [ Aug 28 02:08:49 Method "start" exited with status 0. ]
4 [ Aug 28 02:08:49 Rereading configuration. ]
5 [ Aug 28 02:08:49 No 'refresh' method defined. Treating as :true. ]
6 note: configured to log to "/dev/stdout"
72024-08-28T02:08:49.411ZINFOSledAgent: Ensuring contract reaper thread file = illumos-utils/src/running_zone.rs:113
82024-08-28T02:08:49.411ZINFOSledAgent (BootstrapAgentStartup): finding links ["net0", "net1"] file = sled-agent/src/bootstrap/pre_server.rs:255
92024-08-28T02:08:49.463ZINFOSledAgent (BootstrapAgentStartup): Starting mg-ddm service addr-objs = [AddrObject { interface: "net0", name: "ll" }, AddrObject { interface: "net1", name: "ll" }] file = sled-agent/src/bootstrap/pre_server.rs:262
102024-08-28T02:08:49.463ZINFOSledAgent (BootstrapAgentStartup): Importing mg-ddm service file = sled-agent/src/bootstrap/maghemite.rs:42 path = /opt/oxide/mg-ddm/pkg/ddm/manifest.xml
112024-08-28T02:08:49.576ZINFOSledAgent (BootstrapAgentStartup): Setting mg-ddm interfaces file = sled-agent/src/bootstrap/maghemite.rs:50 interfaces = ("net0/ll" "net1/ll")
122024-08-28T02:08:49.585ZINFOSledAgent (BootstrapAgentStartup): Enabling mg-ddm service file = sled-agent/src/bootstrap/maghemite.rs:56
132024-08-28T02:08:49.592ZINFOSledAgent (BootstrapAgentStartup): Ensuring zfs key directory exists file = sled-agent/src/bootstrap/pre_server.rs:272 path = /var/run/oxide/
142024-08-28T02:08:50.194ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fdb0:18c0:4d0d:a02a::, width: 64 }
152024-08-28T02:08:50.333ZINFOSledAgent: Starting KeyManager file = sled-agent/src/long_running_tasks.rs:126
162024-08-28T02:08:50.333ZINFOSledAgent: Starting StorageManager file = sled-agent/src/long_running_tasks.rs:138
172024-08-28T02:08:50.334ZINFOSledAgent: Starting StorageMonitor file = sled-agent/src/long_running_tasks.rs:151
182024-08-28T02:08:50.334ZINFOSledAgent: Starting HardwareManager file = sled-agent/src/long_running_tasks.rs:173 nongimlet_observed_disks = [] sled_mode = Scrimlet { asic: SoftNpuZone }
192024-08-28T02:08:50.334ZINFOSledAgent (HardwareManager): Creating HardwareManager file = sled-hardware/src/illumos/mod.rs:716
202024-08-28T02:08:50.334ZINFOSledAgent (DumpSetup-worker): DumpSetup poll loop started. file = sled-agent/src/dump_setup.rs:520
212024-08-28T02:08:50.337ZINFOSledAgent (HardwareManager): Generated i86pc baseboard Pc { identifier: "sock", model: "i86pc" } file = sled-hardware/src/illumos/mod.rs:623
222024-08-28T02:08:50.337ZWARNSledAgent (HardwareManager): Device is not a Gimlet (i86pc), proceeding with null hardware view file = sled-hardware/src/illumos/mod.rs:765
232024-08-28T02:08:50.337ZINFOSledAgent: Starting HardwareMonitor file = sled-agent/src/long_running_tasks.rs:187
242024-08-28T02:08:50.337ZINFOSledAgent (HardwareManager): Monitoring for hardware updates file = sled-hardware/src/illumos/mod.rs:824
252024-08-28T02:08:50.337ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/m2_0.vdev
262024-08-28T02:08:50.337ZINFOSledAgent (HardwareMonitor): Checking current full hardware snapshot disks = {} file = sled-agent/src/hardware_monitor.rs:271 underlay_network_info = None
272024-08-28T02:08:50.337ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:406
282024-08-28T02:08:50.337ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
292024-08-28T02:08:50.337ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } path = /scratch/m2_0.vdev
302024-08-28T02:08:50.347ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/m2_0.vdev file = sled-hardware/src/disk.rs:360
312024-08-28T02:08:50.347ZINFOSledAgent (StorageResources): Formatting zpool with generated ID file = sled-hardware/src/disk.rs:371 id = 3dd1ecc3-6cd1-48fe-8e10-622a58ff4257 (zpool)
322024-08-28T02:08:50.444ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:307 zpool = ZpoolName { id: 3dd1ecc3-6cd1-48fe-8e10-622a58ff4257 (zpool), kind: Internal }
332024-08-28T02:08:50.638ZINFOSledAgent (StorageManager): Received DetectedRawDisksChanged { raw_disks: {}, tx: ... } file = sled-storage/src/manager.rs:406
342024-08-28T02:08:50.638ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/m2_1.vdev
352024-08-28T02:08:50.638ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
362024-08-28T02:08:50.638ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
372024-08-28T02:08:50.638ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:406
382024-08-28T02:08:50.638ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
392024-08-28T02:08:50.638ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" } path = /scratch/m2_1.vdev
402024-08-28T02:08:50.646ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/m2_1.vdev file = sled-hardware/src/disk.rs:360
412024-08-28T02:08:50.646ZINFOSledAgent (StorageResources): Formatting zpool with generated ID file = sled-hardware/src/disk.rs:371 id = fadb702a-2570-430e-a63c-e4ed78dd68d2 (zpool)
422024-08-28T02:08:50.730ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:307 zpool = ZpoolName { id: fadb702a-2570-430e-a63c-e4ed78dd68d2 (zpool), kind: Internal }
432024-08-28T02:08:50.917ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_0.vdev
442024-08-28T02:08:50.917ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 3dd1ecc3-6cd1-48fe-8e10-622a58ff4257 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: fadb702a-2570-430e-a63c-e4ed78dd68d2 (zpool), kind: Internal } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
452024-08-28T02:08:50.917ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
462024-08-28T02:08:50.917ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:406
472024-08-28T02:08:50.917ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
482024-08-28T02:08:50.917ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
492024-08-28T02:08:50.917ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 3dd1ecc3-6cd1-48fe-8e10-622a58ff4257 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: fadb702a-2570-430e-a63c-e4ed78dd68d2 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
502024-08-28T02:08:50.917ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_1.vdev
512024-08-28T02:08:50.917ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
522024-08-28T02:08:50.917ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
532024-08-28T02:08:50.917ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:406
542024-08-28T02:08:50.917ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
552024-08-28T02:08:50.917ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_2.vdev
562024-08-28T02:08:50.917ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 3dd1ecc3-6cd1-48fe-8e10-622a58ff4257 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: fadb702a-2570-430e-a63c-e4ed78dd68d2 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
572024-08-28T02:08:50.917ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
582024-08-28T02:08:50.917ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
592024-08-28T02:08:50.917ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:406
602024-08-28T02:08:50.917ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
612024-08-28T02:08:50.917ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_3.vdev
622024-08-28T02:08:50.917ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:406
632024-08-28T02:08:50.917ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 3dd1ecc3-6cd1-48fe-8e10-622a58ff4257 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: fadb702a-2570-430e-a63c-e4ed78dd68d2 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
642024-08-28T02:08:50.917ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
652024-08-28T02:08:50.917ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
662024-08-28T02:08:50.917ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
672024-08-28T02:08:50.917ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_4.vdev
682024-08-28T02:08:50.917ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 3dd1ecc3-6cd1-48fe-8e10-622a58ff4257 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: fadb702a-2570-430e-a63c-e4ed78dd68d2 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
692024-08-28T02:08:50.917ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
702024-08-28T02:08:50.917ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
712024-08-28T02:08:50.917ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:406
722024-08-28T02:08:50.917ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
732024-08-28T02:08:50.917ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_5.vdev
742024-08-28T02:08:50.917ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 3dd1ecc3-6cd1-48fe-8e10-622a58ff4257 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: fadb702a-2570-430e-a63c-e4ed78dd68d2 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
752024-08-28T02:08:50.917ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
762024-08-28T02:08:50.917ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
772024-08-28T02:08:50.917ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:406
782024-08-28T02:08:50.917ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
792024-08-28T02:08:50.917ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_6.vdev
802024-08-28T02:08:50.917ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 3dd1ecc3-6cd1-48fe-8e10-622a58ff4257 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: fadb702a-2570-430e-a63c-e4ed78dd68d2 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
812024-08-28T02:08:50.917ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:406
822024-08-28T02:08:50.917ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
832024-08-28T02:08:50.917ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
842024-08-28T02:08:50.917ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
852024-08-28T02:08:50.917ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_7.vdev
862024-08-28T02:08:50.917ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:406
872024-08-28T02:08:50.917ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
882024-08-28T02:08:50.917ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 3dd1ecc3-6cd1-48fe-8e10-622a58ff4257 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: fadb702a-2570-430e-a63c-e4ed78dd68d2 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
892024-08-28T02:08:50.917ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
902024-08-28T02:08:50.917ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
912024-08-28T02:08:50.917ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_8.vdev
922024-08-28T02:08:50.917ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }), tx: ... } file = sled-storage/src/manager.rs:406
932024-08-28T02:08:50.917ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 3dd1ecc3-6cd1-48fe-8e10-622a58ff4257 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: fadb702a-2570-430e-a63c-e4ed78dd68d2 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
942024-08-28T02:08:50.917ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
952024-08-28T02:08:50.917ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
962024-08-28T02:08:50.917ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
972024-08-28T02:08:50.917ZINFOSledAgent: Waiting for boot disk file = sled-agent/src/long_running_tasks.rs:97
982024-08-28T02:08:50.917ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 3dd1ecc3-6cd1-48fe-8e10-622a58ff4257 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: fadb702a-2570-430e-a63c-e4ed78dd68d2 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
992024-08-28T02:08:50.917ZINFOSledAgent: Found boot disk DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } file = sled-agent/src/long_running_tasks.rs:99
1002024-08-28T02:08:50.917ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
1012024-08-28T02:08:50.917ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
1022024-08-28T02:08:50.917ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
1032024-08-28T02:08:50.917ZINFOSledAgent: Starting Bootstore file = sled-agent/src/long_running_tasks.rs:211
1042024-08-28T02:08:50.917ZINFOSledAgent (bootstore): No ledger in /pool/int/3dd1ecc3-6cd1-48fe-8e10-622a58ff4257/cluster/bootstore-fsm-state.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1052024-08-28T02:08:50.917ZINFOSledAgent (bootstore): No ledger in /pool/int/fadb702a-2570-430e-a63c-e4ed78dd68d2/cluster/bootstore-fsm-state.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1062024-08-28T02:08:50.917ZINFOSledAgent (bootstore): No ledger found. Loading Fsm::State as Uninitialized file = bootstore/src/schemes/v0/storage.rs:83 peer_id = pc-sock-i86pc
1072024-08-28T02:08:50.917ZINFOSledAgent (bootstore): No ledger in /pool/int/3dd1ecc3-6cd1-48fe-8e10-622a58ff4257/cluster/bootstore-network-config.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1082024-08-28T02:08:50.917ZINFOSledAgent (bootstore): No ledger in /pool/int/fadb702a-2570-430e-a63c-e4ed78dd68d2/cluster/bootstore-network-config.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1092024-08-28T02:08:50.917ZINFOSledAgent (bootstore): No ledger found for network config file = bootstore/src/schemes/v0/storage.rs:146 peer_id = pc-sock-i86pc
1102024-08-28T02:08:50.917ZINFOSledAgent: Starting Bootstore DDMD poller file = sled-agent/src/long_running_tasks.rs:216
1112024-08-28T02:08:50.917ZINFOSledAgent: Starting ZoneBundler related tasks file = sled-agent/src/long_running_tasks.rs:231
1122024-08-28T02:08:50.928ZINFOSledAgent (ServiceManager): Creating ServiceManager file = sled-agent/src/services.rs:731
1132024-08-28T02:08:50.928ZINFOSledAgent (auto-cleanup-task): top of bundle cleanup loop file = sled-agent/src/zone_bundle.rs:186 next_cleanup = Instant { tv_sec: 1338, tv_nsec: 777541624 } time_to_next_cleanup = 599.999920198s
1142024-08-28T02:08:50.928ZINFOSledAgent (ServiceManager): Ensuring scrimlet services (enabling services) file = sled-agent/src/services.rs:3861
1152024-08-28T02:08:50.928ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
1162024-08-28T02:08:50.928ZINFOSledAgent (BootstrapAgentStartup): No ledger in /pool/int/3dd1ecc3-6cd1-48fe-8e10-622a58ff4257/config/sled-agent-request.json file = common/src/ledger.rs:188
1172024-08-28T02:08:50.928ZINFOSledAgent (BootstrapAgentStartup): No ledger in /pool/int/fadb702a-2570-430e-a63c-e4ed78dd68d2/config/sled-agent-request.json file = common/src/ledger.rs:188
1182024-08-28T02:08:50.932ZINFOSledAgent (dropshot (BootstrapAgent)): listening file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:205 local_addr = [fdb0:18c0:4d0d:a02a::1]:80
1192024-08-28T02:08:50.932ZINFOSledAgent (SledAgentSprocketsServer): Started listening file = sled-agent/src/bootstrap/sprockets_server.rs:44 local_addr = [fdb0:18c0:4d0d:a02a::1]:12346
1202024-08-28T02:08:50.932ZINFOSledAgent (RSS): Injecting RSS configuration: RackInitializeRequest { trust_quorum_peers: None, bootstrap_discovery: OnlyOurs, ntp_servers: [ "0.pool.ntp.org", ], dns_servers: [ 1.1.1.1, 9.9.9.9, ], internal_services_ip_pool_ranges: [ V4( Ipv4Range { first: 10.151.2.164, last: 10.151.2.173, }, ), ], external_dns_ips: [ 10.151.2.164, 10.151.2.165, ], external_dns_zone_name: "oxide.test", external_certificates: "<redacted>", recovery_silo: RecoverySiloConfig { silo_name: Name( "recovery", ), user_name: UserId( "recovery", ), user_password_hash: NewPasswordHash( PasswordHashString { string: "$argon2id$v=19$m=98304,t=13,p=1$RUlWc0ZxaHo0WFdrN0N6ZQ$S8p52j85GPvMhR/ek3GL0el/oProgTwWpHJZ8lsQQoY", encoding: B64, }, ), }, rack_network_config: RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [ PortConfigV2 { routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), nexthop: 10.151.2.100, vlan_id: None, local_pref: None, }, ], addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.2.174, width: 24, }, ), vlan_id: None, }, ], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: None, bgp_peers: [], autoneg: false, lldp: None, }, ], bgp: [], bfd: [], }, allowed_source_ips: Any, } file = sled-agent/src/rack_setup/service.rs:1053
1212024-08-28T02:08:50.932ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = internal-dns/src/resolver.rs:73
1222024-08-28T02:08:50.932ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
1232024-08-28T02:08:50.932ZINFOSledAgent (RSS): No ledger in /pool/int/3dd1ecc3-6cd1-48fe-8e10-622a58ff4257/config/rss-plan-completed.marker file = common/src/ledger.rs:188
1242024-08-28T02:08:50.932ZINFOSledAgent (RSS): No ledger in /pool/int/fadb702a-2570-430e-a63c-e4ed78dd68d2/config/rss-plan-completed.marker file = common/src/ledger.rs:188
1252024-08-28T02:08:50.932ZINFOSledAgent (RSS): RSS configuration has not been fully applied yet file = sled-agent/src/rack_setup/service.rs:1113
1262024-08-28T02:08:50.932ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
1272024-08-28T02:08:50.932ZINFOSledAgent (RSS): No ledger in /pool/int/3dd1ecc3-6cd1-48fe-8e10-622a58ff4257/config/rss-sled-plan.json file = common/src/ledger.rs:188
1282024-08-28T02:08:50.932ZINFOSledAgent (RSS): No ledger in /pool/int/fadb702a-2570-430e-a63c-e4ed78dd68d2/config/rss-sled-plan.json file = common/src/ledger.rs:188
1292024-08-28T02:08:50.932ZINFOSledAgent (RSS): Creating new allocation plan file = sled-agent/src/rack_setup/service.rs:1154
1302024-08-28T02:08:50.932ZINFOSledAgent (RSS): Serializing plan file = sled-agent/src/rack_setup/plan/sled.rs:155
1312024-08-28T02:08:50.932ZINFOSledAgent (RSS): Creating plan for the sled at fdb0:18c0:4d0d:a02a::1 file = sled-agent/src/rack_setup/plan/sled.rs:128
1322024-08-28T02:08:50.932ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
1332024-08-28T02:08:50.932ZINFOSledAgent (RSS): Writing ledger to /pool/int/3dd1ecc3-6cd1-48fe-8e10-622a58ff4257/config/.rss-sled-plan.json.tmp file = common/src/ledger.rs:199
1342024-08-28T02:08:50.932ZINFOSledAgent (RSS): Writing ledger to /pool/int/fadb702a-2570-430e-a63c-e4ed78dd68d2/config/.rss-sled-plan.json.tmp file = common/src/ledger.rs:199
1352024-08-28T02:08:50.933ZINFOSledAgent (RSS): Sled plan written to storage: Plan { rack_id: 0c5c13d7-1d6f-4932-bd82-66253028d19d, sleds: { [fdb0:18c0:4d0d:a02a::1]:12346: StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: 40b49400-5cd0-4d5c-a006-b241a1ca8261, rack_id: 0c5c13d7-1d6f-4932-bd82-66253028d19d, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64, }, }, }, }, }, config: RackInitializeRequest { trust_quorum_peers: None, bootstrap_discovery: OnlyOurs, ntp_servers: [ "0.pool.ntp.org", ], dns_servers: [ 1.1.1.1, 9.9.9.9, ], internal_services_ip_pool_ranges: [ V4( Ipv4Range { first: 10.151.2.164, last: 10.151.2.173, }, ), ], external_dns_ips: [ 10.151.2.164, 10.151.2.165, ], external_dns_zone_name: "oxide.test", external_certificates: "<redacted>", recovery_silo: RecoverySiloConfig { silo_name: Name( "recovery", ), user_name: UserId( "recovery", ), user_password_hash: NewPasswordHash( PasswordHashString { string: "$argon2id$v=19$m=98304,t=13,p=1$RUlWc0ZxaHo0WFdrN0N6ZQ$S8p52j85GPvMhR/ek3GL0el/oProgTwWpHJZ8lsQQoY", encoding: B64, }, ), }, rack_network_config: RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [ PortConfigV2 { routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), nexthop: 10.151.2.100, vlan_id: None, local_pref: None, }, ], addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.2.174, width: 24, }, ), vlan_id: None, }, ], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: None, bgp_peers: [], autoneg: false, lldp: None, }, ], bgp: [], bfd: [], }, allowed_source_ips: Any, }, } file = sled-agent/src/rack_setup/plan/sled.rs:175
1362024-08-28T02:08:50.933ZINFOSledAgent (RSS): Writing Rack Network Configuration to bootstore file = sled-agent/src/rack_setup/service.rs:1187
1372024-08-28T02:08:50.933ZINFOSledAgent (bootstore): Attempting to update network config with generation: 1, current_generation: 0 file = bootstore/src/schemes/v0/peer.rs:564 peer_id = pc-sock-i86pc
1382024-08-28T02:08:50.933ZINFOSledAgent (bootstore): Writing ledger to /pool/int/3dd1ecc3-6cd1-48fe-8e10-622a58ff4257/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
1392024-08-28T02:08:50.933ZINFOSledAgent (bootstore): Writing ledger to /pool/int/fadb702a-2570-430e-a63c-e4ed78dd68d2/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
1402024-08-28T02:08:50.933ZINFOSledAgent (bootstore): Broadcasting network config with generation 1 file = bootstore/src/schemes/v0/peer.rs:648 peer_id = pc-sock-i86pc
1412024-08-28T02:08:50.933ZINFOSledAgent (BootstrapAgentRssHandler): Received initialization request from RSS file = sled-agent/src/bootstrap/rss_handle.rs:217 request = StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: 40b49400-5cd0-4d5c-a006-b241a1ca8261, rack_id: 0c5c13d7-1d6f-4932-bd82-66253028d19d, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64 } } } } target_sled = [fdb0:18c0:4d0d:a02a::1]:12346
1422024-08-28T02:08:50.933ZINFOSledAgent (SledAgentSprocketsServer): Accepted connection file = sled-agent/src/bootstrap/sprockets_server.rs:66 remote_addr = [fdb0:18c0:4d0d:a02a::1]:36814
1432024-08-28T02:08:50.933ZINFOSledAgent (SledAgentMain): Loading Sled Agent: StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: 40b49400-5cd0-4d5c-a006-b241a1ca8261, rack_id: 0c5c13d7-1d6f-4932-bd82-66253028d19d, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64 } } } } file = sled-agent/src/bootstrap/server.rs:359
1442024-08-28T02:08:50.933ZINFOSledAgent (SledAgentMain): KeyManager: using hardcoded secret retriever file = sled-agent/src/bootstrap/server.rs:374
1452024-08-28T02:08:50.933ZINFOSledAgent: setting up sled agent server file = sled-agent/src/server.rs:44
1462024-08-28T02:08:50.933ZINFOSledAgent (StorageManager): Received KeyManagerReady file = sled-storage/src/manager.rs:406
1472024-08-28T02:08:50.933ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = internal-dns/src/resolver.rs:73
1482024-08-28T02:08:50.933ZINFOSledAgent (StorageManager): Loading storage config file = sled-storage/src/manager.rs:574
1492024-08-28T02:08:50.933ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:101::, width: 64 }
1502024-08-28T02:08:50.933ZINFOSledAgent (StorageManager): No ledger in /pool/int/3dd1ecc3-6cd1-48fe-8e10-622a58ff4257/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = load_ledger
1512024-08-28T02:08:50.933ZINFOSledAgent (StorageManager): No ledger in /pool/int/fadb702a-2570-430e-a63c-e4ed78dd68d2/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = load_ledger
1522024-08-28T02:08:50.933ZINFOSledAgent (StorageManager): No ledger of physical disks exists file = sled-storage/src/manager.rs:563
1532024-08-28T02:08:50.933ZINFOSledAgent (StorageManager): KeyManager ready, but no ledger detected file = sled-storage/src/manager.rs:591
1542024-08-28T02:08:50.933ZINFOSledAgent (StorageManager): automatically managing disks file = sled-storage/src/manager.rs:395
1552024-08-28T02:08:50.934ZINFOSledAgent (StorageResources): Synchronizing disk managment file = sled-storage/src/resources.rs:302
1562024-08-28T02:08:50.934ZINFOSledAgent (StorageManager): Successfully synchronized disks without error file = sled-storage/src/manager.rs:481 result = DisksManagementResult { status: [] }
1572024-08-28T02:08:50.935ZINFOSledAgent (ServiceManager): Enabling switch zone (new) file = sled-agent/src/services.rs:4158
1582024-08-28T02:08:50.936ZINFOSledAgent (ServiceManager): Starting switch zone file = sled-agent/src/services.rs:4540
1592024-08-28T02:08:50.966ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
1602024-08-28T02:08:50.987ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_switch file = illumos-utils/src/zone.rs:316
1612024-08-28T02:08:51.003ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_switch file = illumos-utils/src/zone.rs:354
1622024-08-28T02:08:51.049ZINFOSledAgent: SledAgent::new(..) starting file = sled-agent/src/sled_agent.rs:385 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261
1632024-08-28T02:08:51.050ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
1642024-08-28T02:08:51.050ZINFOSledAgent: Requested swap device of size 64 GiB file = sled-agent/src/sled_agent.rs:397 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261
1652024-08-28T02:08:51.050ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1662024-08-28T02:08:51.053ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 84.226389ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "d894179e-cbcf-4869-a4fe-6bed6970b064", "content-length": "124", "date": "Wed, 28 Aug 2024 02:08:50 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "d894179e-cbcf-4869-a4fe-6bed6970b064" } file = clients/ddm-admin-client/src/lib.rs:177
1672024-08-28T02:08:51.063ZINFOSledAgent: attempting to create encrypted zvol: name="oxi_3dd1ecc3-6cd1-48fe-8e10-622a58ff4257/swap", size_gb=64 file = sled-agent/src/swap_device.rs:153
1682024-08-28T02:08:51.088ZINFOSledAgent: successfully created encrypted zvol: name="oxi_3dd1ecc3-6cd1-48fe-8e10-622a58ff4257/swap", size_gb=64 file = sled-agent/src/swap_device.rs:249
1692024-08-28T02:08:51.088ZINFOSledAgent: adding swap device: swapname="/dev/zvol/dsk/oxi_3dd1ecc3-6cd1-48fe-8e10-622a58ff4257/swap" file = sled-agent/src/swap_device.rs:96
1702024-08-28T02:08:51.094ZINFOSledAgent: Mounting backing filesystems file = sled-agent/src/sled_agent.rs:412 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261
1712024-08-28T02:08:51.094ZINFOSledAgent (BackingFs): Processing fmd file = sled-agent/src/backing_fs.rs:126
1722024-08-28T02:08:51.094ZINFOSledAgent (BackingFs): Ensuring dataset oxi_3dd1ecc3-6cd1-48fe-8e10-622a58ff4257/backing/fmd file = sled-agent/src/backing_fs.rs:136
1732024-08-28T02:08:51.138ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1742024-08-28T02:08:51.138ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 87.425766ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "e2962582-9419-469d-bf59-1781c0f1a71b", "content-length": "124", "date": "Wed, 28 Aug 2024 02:08:50 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "e2962582-9419-469d-bf59-1781c0f1a71b" } file = clients/ddm-admin-client/src/lib.rs:177
1752024-08-28T02:08:51.169ZINFOSledAgent (BackingFs): Stopping service svc:/system/fmd:default file = sled-agent/src/backing_fs.rs:170
1762024-08-28T02:08:51.227ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1772024-08-28T02:08:51.227ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 125.59909ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "3396fafb-31f2-4925-af4f-fc0f9f4f53d4", "content-length": "124", "date": "Wed, 28 Aug 2024 02:08:51 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "3396fafb-31f2-4925-af4f-fc0f9f4f53d4" } file = clients/ddm-admin-client/src/lib.rs:177
1782024-08-28T02:08:51.354ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1792024-08-28T02:08:51.354ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 258.055655ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "a6c40460-aa36-4056-9f7c-335af33342ab", "content-length": "124", "date": "Wed, 28 Aug 2024 02:08:51 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "a6c40460-aa36-4056-9f7c-335af33342ab" } file = clients/ddm-admin-client/src/lib.rs:177
1802024-08-28T02:08:51.388ZINFOSledAgent (BackingFs): Mounting oxi_3dd1ecc3-6cd1-48fe-8e10-622a58ff4257/backing/fmd on /var/fm/fmd file = sled-agent/src/backing_fs.rs:178
1812024-08-28T02:08:51.401ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/rsrc file = sled-agent/src/backing_fs.rs:186
1822024-08-28T02:08:51.401ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/ckpt file = sled-agent/src/backing_fs.rs:186
1832024-08-28T02:08:51.401ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/xprt file = sled-agent/src/backing_fs.rs:186
1842024-08-28T02:08:51.401ZINFOSledAgent (BackingFs): Starting service svc:/system/fmd:default file = sled-agent/src/backing_fs.rs:192
1852024-08-28T02:08:51.614ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1862024-08-28T02:08:51.614ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 152.068742ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "d42e4a8a-fa6c-435e-b5e2-a64f461fd93c", "content-length": "124", "date": "Wed, 28 Aug 2024 02:08:51 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "d42e4a8a-fa6c-435e-b5e2-a64f461fd93c" } file = clients/ddm-admin-client/src/lib.rs:177
1872024-08-28T02:08:51.767ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1882024-08-28T02:08:51.768ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 156.083654ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "1a5dca4f-3c0d-48d4-b8b3-6150d8d54107", "content-length": "124", "date": "Wed, 28 Aug 2024 02:08:51 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "1a5dca4f-3c0d-48d4-b8b3-6150d8d54107" } file = clients/ddm-admin-client/src/lib.rs:177
1892024-08-28T02:08:51.926ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1902024-08-28T02:08:51.927ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 394.282119ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "58f5359c-47d8-46f7-b2b0-f8a7fa2f14df", "content-length": "124", "date": "Wed, 28 Aug 2024 02:08:51 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "58f5359c-47d8-46f7-b2b0-f8a7fa2f14df" } file = clients/ddm-admin-client/src/lib.rs:177
1912024-08-28T02:08:52.141ZINFOSledAgent: using '[AddrObject { interface: "net0", name: "ll" }, AddrObject { interface: "net1", name: "ll" }]' as data links for xde driver file = illumos-utils/src/opte/illumos.rs:90 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261
1922024-08-28T02:08:52.147ZINFOSledAgent (dropshot): listening file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:205 local_addr = [fd00:1122:3344:101::1]:49250 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261
1932024-08-28T02:08:52.147ZINFOSledAgent (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = internal-dns/src/resolver.rs:73 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261
1942024-08-28T02:08:52.147ZINFOSledAgent (producer-server): starting oximeter metric producer server address = [fd00:1122:3344:101::1]:49250 file = oximeter/producer/src/lib.rs:283 interval = 30s producer_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261
1952024-08-28T02:08:52.148ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 207.4118ms error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261
1962024-08-28T02:08:52.149ZINFOSledAgent (VmmReservoirManager): 50% of 102952026112 physical ram = 51476013056 bytes) file = sled-agent/src/vmm_reservoir.rs:267 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261
1972024-08-28T02:08:52.149ZINFOSledAgent (VmmReservoirManager): Setting reservoir size to 49090 MiB bytes file = sled-agent/src/vmm_reservoir.rs:275 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261
1982024-08-28T02:08:52.149ZINFOSledAgent (ServiceManager): sled agent started file = sled-agent/src/services.rs:915 underlay_address = fd00:1122:3344:101::1
1992024-08-28T02:08:52.149ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = internal-dns/src/resolver.rs:73
2002024-08-28T02:08:52.322ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
2012024-08-28T02:08:52.358ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 671.740372ms error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261
2022024-08-28T02:08:53.032ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 616.728789ms error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261
2032024-08-28T02:08:53.650ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 1.251994198s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261
2042024-08-28T02:08:54.569ZINFOSledAgent (ServiceManager): Setting up dendrite service file = sled-agent/src/services.rs:2634
2052024-08-28T02:08:54.569ZINFOSledAgent (ServiceManager): Setting up lldpd service file = sled-agent/src/services.rs:2852
2062024-08-28T02:08:54.569ZINFOSledAgent (ServiceManager): Setting up MGS service file = sled-agent/src/services.rs:2504
2072024-08-28T02:08:54.569ZINFOSledAgent (ServiceManager): Setting up wicketd service file = sled-agent/src/services.rs:2551
2082024-08-28T02:08:54.569ZINFOSledAgent (ServiceManager): Setting up mgd service file = sled-agent/src/services.rs:2933
2092024-08-28T02:08:54.569ZINFOSledAgent (ServiceManager): Setting up mg-ddm service file = sled-agent/src/services.rs:2975
2102024-08-28T02:08:54.571ZINFOSledAgent (ServiceManager): Setting up tfport service file = sled-agent/src/services.rs:2777
2112024-08-28T02:08:54.572ZINFOSledAgent (ServiceManager): Setting up Simulated SP service file = sled-agent/src/services.rs:2545
2122024-08-28T02:08:54.572ZINFOSledAgent (ServiceManager): Profile for oxz_switch: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService0'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/mgs"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="address" value='[::1]:12225'/> <propval type="astring" name="id" value='7ab6cf11-bb3f-4b6e-b7eb-0005add95826'/> <propval type="astring" name="rack_id" value='0c5c13d7-1d6f-4932-bd82-66253028d19d'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/wicketd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="address" value='[::1]:12226'/> <propval type="astring" name="artifact-address" value='[fdb0:18c0:4d0d:a02a::2]:12227'/> <propval type="astring" name="baseboard-file" value='/opt/oxide/baseboard.json'/> <propval type="astring" name="mgs-address" value='[::1]:12225'/> <propval type="astring" name="nexus-proxy-address" value='[::]:12229'/> <propval type="astring" name="rack-subnet" value='fd00:1122:3344::'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/switch_zone_setup"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="gz_local_link_addr" value='fe80::8:20ff:fe2c:7eb5'/> <property type="astring" name="link_local_links"> <astring_list> <value_node value="tfportrear0_0" /> <value_node value="oxBootstrap0" /> </astring_list> </property> <propval type="astring" name="bootstrap_addr" value='fdb0:18c0:4d0d:a02a::2'/> <propval type="astring" name="bootstrap_vnic" value='oxBootstrap0'/> <propval type="astring" name="baseboard_info" value='{"type":"pc","identifier":"sock","model":"i86pc"}'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/dendrite"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="sled_id" value='40b49400-5cd0-4d5c-a006-b241a1ca8261'/> <propval type="astring" name="rack_id" value='0c5c13d7-1d6f-4932-bd82-66253028d19d'/> <propval type="astring" name="address" value='[::1]:12224'/> <propval type="astring" name="front_ports" value='1'/> <propval type="astring" name="rear_ports" value='1'/> <propval type="astring" name="port_config" value='/opt/oxide/dendrite/misc/softnpu_single_sled_config.toml'/> <propval type="astring" name="mgmt" value='uds'/> <propval type="astring" name="uds_path" value='/opt/softnpu/stuff'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/tfport"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="host" value='[::1]'/> <propval type="astring" name="port" value='12224'/> <propval type="astring" name="flags" value='--sync-only'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/lldpd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="board_rev" value='softnpu_front_1_rear_1'/> <propval type="astring" name="scrimlet_id" value='sock'/> <propval type="astring" name="scrimlet_model" value='i86pc'/> <propval type="astring" name="address" value='[::1]:12230'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/pumpkind"> </service> <service version="1" type="service" name="oxide/mgd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="sled_uuid" value='40b49400-5cd0-4d5c-a006-b241a1ca8261'/> <propval type="astring" name="rack_uuid" value='0c5c13d7-1d6f-4932-bd82-66253028d19d'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/mg-ddm"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="mode" value='transit'/> <propval type="astring" name="dendrite" value='true'/> <propval type="astring" name="sled_uuid" value='40b49400-5cd0-4d5c-a006-b241a1ca8261'/> <propval type="astring" name="rack_uuid" value='0c5c13d7-1d6f-4932-bd82-66253028d19d'/> <propval type="astring" name="interfaces" value='tfportrear0_0/ll'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/uplink"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
2132024-08-28T02:08:54.572ZINFOSledAgent (ServiceManager): Booting oxz_switch zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_switch
2142024-08-28T02:08:54.895ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 52.477921ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2152024-08-28T02:08:54.904ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 2.200279941s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261
2162024-08-28T02:08:55.078ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 145.54661ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2172024-08-28T02:08:55.253ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 282.938041ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2182024-08-28T02:08:55.602ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 359.447817ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2192024-08-28T02:08:56.178ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 1.040870068s file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2202024-08-28T02:08:57.106ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 11.264254601s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261
2212024-08-28T02:08:57.293ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 1.41163268s file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2222024-08-28T02:08:58.889ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 507.907656ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2232024-08-28T02:08:59.586ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 519.416616ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2242024-08-28T02:09:00.587ZINFOSledAgent (dropshot (SledAgent)): listening file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:205 local_addr = [fd00:1122:3344:101::1]:12345
2252024-08-28T02:09:00.587ZINFOSledAgent (SledAgentMain): Sled Agent loaded; recording configuration file = sled-agent/src/bootstrap/server.rs:429
2262024-08-28T02:09:00.587ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
2272024-08-28T02:09:00.587ZINFOSledAgent (SledAgentMain): Writing ledger to /pool/int/3dd1ecc3-6cd1-48fe-8e10-622a58ff4257/config/.sled-agent-request.json.tmp file = common/src/ledger.rs:199
2282024-08-28T02:09:00.587ZINFOSledAgent (SledAgentMain): Writing ledger to /pool/int/fadb702a-2570-430e-a63c-e4ed78dd68d2/config/.sled-agent-request.json.tmp file = common/src/ledger.rs:199
2292024-08-28T02:09:00.587ZINFOSledAgent (HardwareMonitor): Sled Agent Started file = sled-agent/src/hardware_monitor.rs:138
2302024-08-28T02:09:00.588ZINFOSledAgent (HardwareMonitor): Checking current full hardware snapshot disks = {} file = sled-agent/src/hardware_monitor.rs:271 underlay_network_info = Some((fd00:1122:3344:101::2, Some(RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56 }, infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [PortConfigV2 { routes: [RouteConfig { destination: V4(Ipv4Net { addr: 0.0.0.0, width: 0 }), nexthop: 10.151.2.100, vlan_id: None, local_pref: None }], addresses: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: None, bgp_peers: [], autoneg: false, lldp: None }], bgp: [], bfd: [] })))
2312024-08-28T02:09:00.588ZINFOSledAgent (ServiceManager): Ensuring scrimlet services (enabling services) file = sled-agent/src/services.rs:3861
2322024-08-28T02:09:00.588ZINFOSledAgent (SledAgentSprocketsServer): Connection closed file = sled-agent/src/bootstrap/sprockets_server.rs:73 remote_addr = [fdb0:18c0:4d0d:a02a::1]:36814
2332024-08-28T02:09:00.843ZINFOSledAgent (BootstrapAgentRssHandler): Peer agent initialized file = sled-agent/src/bootstrap/rss_handle.rs:108 peer = [fdb0:18c0:4d0d:a02a::1]:12346
2342024-08-28T02:09:00.843ZINFOSledAgent (BootstrapAgentRssHandler): Initialized sled agent file = sled-agent/src/bootstrap/rss_handle.rs:232 target_sled = [fdb0:18c0:4d0d:a02a::1]:12346
2352024-08-28T02:09:00.843ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
2362024-08-28T02:09:00.843ZINFOSledAgent (RSS): No ledger in /pool/int/3dd1ecc3-6cd1-48fe-8e10-622a58ff4257/config/rss-service-plan-v4.json file = common/src/ledger.rs:188
2372024-08-28T02:09:00.843ZINFOSledAgent (RSS): No ledger in /pool/int/fadb702a-2570-430e-a63c-e4ed78dd68d2/config/rss-service-plan-v4.json file = common/src/ledger.rs:188
2382024-08-28T02:09:00.843ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
2392024-08-28T02:09:00.843ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
2402024-08-28T02:09:00.843ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
2412024-08-28T02:09:00.848ZINFOSledAgent (ServiceManager): Re-enabling running switch zone (new address) file = sled-agent/src/services.rs:4182 new = [fd00:1122:3344:101::2, ::1] old = [::1]
2422024-08-28T02:09:00.848ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::2 exists file = sled-agent/src/services.rs:4198
2432024-08-28T02:09:00.848ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:101::2, prefix: 64 })) file = illumos-utils/src/running_zone.rs:534 zone = oxz_switch
2442024-08-28T02:09:01.110ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:44150
2452024-08-28T02:09:01.110ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
2462024-08-28T02:09:01.110ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 194 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:44150 req_id = 96174ea4-2149-41fc-aee3-30640e615932 response_code = 200 uri = /inventory
2472024-08-28T02:09:01.142ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:57419
2482024-08-28T02:09:01.142ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 40 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:57419 req_id = 10a1ca66-75fc-435c-83ab-bb204a6afc53 response_code = 200 uri = /sled-role
2492024-08-28T02:09:01.142ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
2502024-08-28T02:09:01.142ZINFOSledAgent (RSS): Writing ledger to /pool/int/3dd1ecc3-6cd1-48fe-8e10-622a58ff4257/config/.rss-service-plan-v4.json.tmp file = common/src/ledger.rs:199
2512024-08-28T02:09:01.143ZINFOSledAgent (RSS): Writing ledger to /pool/int/fadb702a-2570-430e-a63c-e4ed78dd68d2/config/.rss-service-plan-v4.json.tmp file = common/src/ledger.rs:199
2522024-08-28T02:09:01.143ZINFOSledAgent (RSS): Service plan written to storage file = sled-agent/src/rack_setup/plan/service.rs:998
2532024-08-28T02:09:01.176ZINFOSledAgent (RSS): attempting to set up sled's storage: OmicronPhysicalDisksConfig { generation: Generation(1), disks: [OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, id: 5335d373-0821-4d96-bb3c-e55af9dbf8d4, pool_id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, id: 463b6b9a-9a54-4d78-8465-1e88f5abb04e, pool_id: 79e9b421-598b-4117-bcfb-c4abc62898c1 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, id: edd28b8b-7c4c-416a-ad52-7bd91814a7a8, pool_id: ae881951-d9c0-4515-a6b0-7e5c2297d58f (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, id: c0a9c5fa-be04-4360-bda8-201b7158aa46, pool_id: 33490a89-0fb3-432e-bbc7-3c605249772c (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, id: 39a6e4f2-c58c-4701-9d65-0cd05ab5e2a6, pool_id: 9b46afde-44d9-45f0-a9fc-f4fa353c3113 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, id: 88d9b634-389c-43c0-92d4-3bf1cddf2e5a, pool_id: 5d8fb525-afb0-4464-ad9b-8e50ab9d715a (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, id: bd8d5cd6-4e35-413f-8fdf-6ba7652ffcf3, pool_id: 95dac2cb-734b-49b1-8778-a286d737d1fa (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, id: cc0bfba1-13ee-4ed6-b869-e6d9ec322c1c, pool_id: 37700612-da7c-4ef5-9910-6b71dd793192 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, id: 040a525f-6967-4b40-b4ff-2088b4636246, pool_id: e56ec7bf-ebda-425c-b581-1265bb74473b (zpool) }] } file = sled-agent/src/rack_setup/service.rs:355 sled_address = [fd00:1122:3344:101::1]:12345
2542024-08-28T02:09:01.176ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:51271
2552024-08-28T02:09:01.176ZINFOSledAgent: physical disks ensure file = sled-agent/src/sled_agent.rs:830 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261
2562024-08-28T02:09:01.176ZINFOSledAgent (StorageManager): Received OmicronPhysicalDisksEnsure { config: OmicronPhysicalDisksConfig { generation: Generation(1), disks: [OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, id: 5335d373-0821-4d96-bb3c-e55af9dbf8d4, pool_id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, id: 463b6b9a-9a54-4d78-8465-1e88f5abb04e, pool_id: 79e9b421-598b-4117-bcfb-c4abc62898c1 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, id: edd28b8b-7c4c-416a-ad52-7bd91814a7a8, pool_id: ae881951-d9c0-4515-a6b0-7e5c2297d58f (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, id: c0a9c5fa-be04-4360-bda8-201b7158aa46, pool_id: 33490a89-0fb3-432e-bbc7-3c605249772c (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, id: 39a6e4f2-c58c-4701-9d65-0cd05ab5e2a6, pool_id: 9b46afde-44d9-45f0-a9fc-f4fa353c3113 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, id: 88d9b634-389c-43c0-92d4-3bf1cddf2e5a, pool_id: 5d8fb525-afb0-4464-ad9b-8e50ab9d715a (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, id: bd8d5cd6-4e35-413f-8fdf-6ba7652ffcf3, pool_id: 95dac2cb-734b-49b1-8778-a286d737d1fa (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, id: cc0bfba1-13ee-4ed6-b869-e6d9ec322c1c, pool_id: 37700612-da7c-4ef5-9910-6b71dd793192 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, id: 040a525f-6967-4b40-b4ff-2088b4636246, pool_id: e56ec7bf-ebda-425c-b581-1265bb74473b (zpool) }] }, tx: ... } file = sled-storage/src/manager.rs:406
2572024-08-28T02:09:01.176ZINFOSledAgent (StorageManager): No ledger in /pool/int/3dd1ecc3-6cd1-48fe-8e10-622a58ff4257/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = omicron_physical_disks_ensure
2582024-08-28T02:09:01.176ZINFOSledAgent (StorageManager): No ledger in /pool/int/fadb702a-2570-430e-a63c-e4ed78dd68d2/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = omicron_physical_disks_ensure
2592024-08-28T02:09:01.176ZINFOSledAgent (StorageManager): No previously-stored 'requested disks', creating new ledger file = sled-storage/src/manager.rs:645 request = omicron_physical_disks_ensure
2602024-08-28T02:09:01.176ZINFOSledAgent (StorageResources): Synchronizing disk managment file = sled-storage/src/resources.rs:302
2612024-08-28T02:09:01.176ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:342
2622024-08-28T02:09:01.176ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
2632024-08-28T02:09:01.176ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } path = /scratch/u2_0.vdev
2642024-08-28T02:09:01.340ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_0.vdev file = sled-hardware/src/disk.rs:360
2652024-08-28T02:09:01.340ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool)
2662024-08-28T02:09:01.589ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/40b49400-5cd0-4d5c-a006-b241a1ca8261): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/probe_manager.rs:230 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261
2672024-08-28T02:09:02.506ZINFOSledAgent (VmmReservoirManager): Finished setting reservoir size to 49090 MiB bytes file = sled-agent/src/vmm_reservoir.rs:279 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261
2682024-08-28T02:09:02.518ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:307 zpool = ZpoolName { id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool), kind: External }
2692024-08-28T02:09:02.526ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:351
2702024-08-28T02:09:02.526ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:353
2712024-08-28T02:09:02.526ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:357
2722024-08-28T02:09:02.526ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:359
2732024-08-28T02:09:02.526ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key")
2742024-08-28T02:09:02.526ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key")
2752024-08-28T02:09:02.526ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:371
2762024-08-28T02:09:02.562ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
2772024-08-28T02:09:02.571ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_fe881465-8582-4eba-8c48-1f5016ef66f5/crypt/zone file = sled-storage/src/dataset.rs:414
2782024-08-28T02:09:02.591ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/40b49400-5cd0-4d5c-a006-b241a1ca8261): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/probe_manager.rs:230 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261
2792024-08-28T02:09:02.694ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_fe881465-8582-4eba-8c48-1f5016ef66f5 file = sled-storage/src/dataset.rs:499
2802024-08-28T02:09:02.702ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:440
2812024-08-28T02:09:02.703ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:357
2822024-08-28T02:09:02.703ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:342
2832024-08-28T02:09:02.703ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
2842024-08-28T02:09:02.703ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } path = /scratch/u2_1.vdev
2852024-08-28T02:09:02.712ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_1.vdev file = sled-hardware/src/disk.rs:360
2862024-08-28T02:09:02.712ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = 79e9b421-598b-4117-bcfb-c4abc62898c1 (zpool)
2872024-08-28T02:09:02.805ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:307 zpool = ZpoolName { id: 79e9b421-598b-4117-bcfb-c4abc62898c1 (zpool), kind: External }
2882024-08-28T02:09:02.812ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:351
2892024-08-28T02:09:02.812ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:353
2902024-08-28T02:09:02.812ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:357
2912024-08-28T02:09:02.812ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:359
2922024-08-28T02:09:02.812ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key")
2932024-08-28T02:09:02.812ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key")
2942024-08-28T02:09:02.812ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:371
2952024-08-28T02:09:02.848ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
2962024-08-28T02:09:02.857ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_79e9b421-598b-4117-bcfb-c4abc62898c1/crypt/zone file = sled-storage/src/dataset.rs:414
2972024-08-28T02:09:02.981ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_79e9b421-598b-4117-bcfb-c4abc62898c1 file = sled-storage/src/dataset.rs:499
2982024-08-28T02:09:02.989ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:440
2992024-08-28T02:09:02.989ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:357
3002024-08-28T02:09:02.989ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:342
3012024-08-28T02:09:02.989ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3022024-08-28T02:09:02.989ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } path = /scratch/u2_2.vdev
3032024-08-28T02:09:02.998ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_2.vdev file = sled-hardware/src/disk.rs:360
3042024-08-28T02:09:02.998ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = ae881951-d9c0-4515-a6b0-7e5c2297d58f (zpool)
3052024-08-28T02:09:03.085ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/dataset.rs:307 zpool = ZpoolName { id: ae881951-d9c0-4515-a6b0-7e5c2297d58f (zpool), kind: External }
3062024-08-28T02:09:03.092ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/dataset.rs:351
3072024-08-28T02:09:03.092ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:353
3082024-08-28T02:09:03.092ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:357
3092024-08-28T02:09:03.092ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:359
3102024-08-28T02:09:03.092ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key")
3112024-08-28T02:09:03.092ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key")
3122024-08-28T02:09:03.092ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:371
3132024-08-28T02:09:03.129ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3142024-08-28T02:09:03.137ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_ae881951-d9c0-4515-a6b0-7e5c2297d58f/crypt/zone file = sled-storage/src/dataset.rs:414
3152024-08-28T02:09:03.258ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_ae881951-d9c0-4515-a6b0-7e5c2297d58f file = sled-storage/src/dataset.rs:499
3162024-08-28T02:09:03.265ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:440
3172024-08-28T02:09:03.265ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:357
3182024-08-28T02:09:03.265ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:342
3192024-08-28T02:09:03.265ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3202024-08-28T02:09:03.265ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } path = /scratch/u2_3.vdev
3212024-08-28T02:09:03.274ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_3.vdev file = sled-hardware/src/disk.rs:360
3222024-08-28T02:09:03.274ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = 33490a89-0fb3-432e-bbc7-3c605249772c (zpool)
3232024-08-28T02:09:03.362ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/dataset.rs:307 zpool = ZpoolName { id: 33490a89-0fb3-432e-bbc7-3c605249772c (zpool), kind: External }
3242024-08-28T02:09:03.369ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/dataset.rs:351
3252024-08-28T02:09:03.369ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:353
3262024-08-28T02:09:03.369ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:357
3272024-08-28T02:09:03.369ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:359
3282024-08-28T02:09:03.369ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key")
3292024-08-28T02:09:03.369ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key")
3302024-08-28T02:09:03.369ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:371
3312024-08-28T02:09:03.406ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3322024-08-28T02:09:03.413ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_33490a89-0fb3-432e-bbc7-3c605249772c/crypt/zone file = sled-storage/src/dataset.rs:414
3332024-08-28T02:09:03.527ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_33490a89-0fb3-432e-bbc7-3c605249772c file = sled-storage/src/dataset.rs:499
3342024-08-28T02:09:03.534ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:440
3352024-08-28T02:09:03.534ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:357
3362024-08-28T02:09:03.534ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:342
3372024-08-28T02:09:03.534ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3382024-08-28T02:09:03.534ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } path = /scratch/u2_4.vdev
3392024-08-28T02:09:03.544ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_4.vdev file = sled-hardware/src/disk.rs:360
3402024-08-28T02:09:03.544ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = 9b46afde-44d9-45f0-a9fc-f4fa353c3113 (zpool)
3412024-08-28T02:09:03.636ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/dataset.rs:307 zpool = ZpoolName { id: 9b46afde-44d9-45f0-a9fc-f4fa353c3113 (zpool), kind: External }
3422024-08-28T02:09:03.643ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/dataset.rs:351
3432024-08-28T02:09:03.643ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:353
3442024-08-28T02:09:03.643ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:357
3452024-08-28T02:09:03.643ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:359
3462024-08-28T02:09:03.643ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key")
3472024-08-28T02:09:03.644ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key")
3482024-08-28T02:09:03.644ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:371
3492024-08-28T02:09:03.647ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/40b49400-5cd0-4d5c-a006-b241a1ca8261): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/probe_manager.rs:230 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261
3502024-08-28T02:09:03.679ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3512024-08-28T02:09:03.686ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_9b46afde-44d9-45f0-a9fc-f4fa353c3113/crypt/zone file = sled-storage/src/dataset.rs:414
3522024-08-28T02:09:03.807ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_9b46afde-44d9-45f0-a9fc-f4fa353c3113 file = sled-storage/src/dataset.rs:499
3532024-08-28T02:09:03.815ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:440
3542024-08-28T02:09:03.815ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:357
3552024-08-28T02:09:03.815ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:342
3562024-08-28T02:09:03.815ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3572024-08-28T02:09:03.815ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } path = /scratch/u2_5.vdev
3582024-08-28T02:09:03.824ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_5.vdev file = sled-hardware/src/disk.rs:360
3592024-08-28T02:09:03.824ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = 5d8fb525-afb0-4464-ad9b-8e50ab9d715a (zpool)
3602024-08-28T02:09:03.912ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/dataset.rs:307 zpool = ZpoolName { id: 5d8fb525-afb0-4464-ad9b-8e50ab9d715a (zpool), kind: External }
3612024-08-28T02:09:03.919ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/dataset.rs:351
3622024-08-28T02:09:03.919ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:353
3632024-08-28T02:09:03.919ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:357
3642024-08-28T02:09:03.919ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:359
3652024-08-28T02:09:03.919ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key")
3662024-08-28T02:09:03.919ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key")
3672024-08-28T02:09:03.919ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:371
3682024-08-28T02:09:03.950ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3692024-08-28T02:09:03.957ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_5d8fb525-afb0-4464-ad9b-8e50ab9d715a/crypt/zone file = sled-storage/src/dataset.rs:414
3702024-08-28T02:09:04.071ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_5d8fb525-afb0-4464-ad9b-8e50ab9d715a file = sled-storage/src/dataset.rs:499
3712024-08-28T02:09:04.078ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:440
3722024-08-28T02:09:04.078ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:357
3732024-08-28T02:09:04.078ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:342
3742024-08-28T02:09:04.078ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3752024-08-28T02:09:04.078ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } path = /scratch/u2_6.vdev
3762024-08-28T02:09:04.088ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_6.vdev file = sled-hardware/src/disk.rs:360
3772024-08-28T02:09:04.088ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = 95dac2cb-734b-49b1-8778-a286d737d1fa (zpool)
3782024-08-28T02:09:04.182ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/dataset.rs:307 zpool = ZpoolName { id: 95dac2cb-734b-49b1-8778-a286d737d1fa (zpool), kind: External }
3792024-08-28T02:09:04.190ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/dataset.rs:351
3802024-08-28T02:09:04.190ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:353
3812024-08-28T02:09:04.190ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:357
3822024-08-28T02:09:04.190ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:359
3832024-08-28T02:09:04.190ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key")
3842024-08-28T02:09:04.190ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key")
3852024-08-28T02:09:04.190ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:371
3862024-08-28T02:09:04.225ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3872024-08-28T02:09:04.233ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_95dac2cb-734b-49b1-8778-a286d737d1fa/crypt/zone file = sled-storage/src/dataset.rs:414
3882024-08-28T02:09:04.362ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_95dac2cb-734b-49b1-8778-a286d737d1fa file = sled-storage/src/dataset.rs:499
3892024-08-28T02:09:04.372ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:440
3902024-08-28T02:09:04.372ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:357
3912024-08-28T02:09:04.372ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:342
3922024-08-28T02:09:04.372ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3932024-08-28T02:09:04.372ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } path = /scratch/u2_7.vdev
3942024-08-28T02:09:04.382ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_7.vdev file = sled-hardware/src/disk.rs:360
3952024-08-28T02:09:04.383ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = 37700612-da7c-4ef5-9910-6b71dd793192 (zpool)
3962024-08-28T02:09:04.479ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/dataset.rs:307 zpool = ZpoolName { id: 37700612-da7c-4ef5-9910-6b71dd793192 (zpool), kind: External }
3972024-08-28T02:09:04.486ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/dataset.rs:351
3982024-08-28T02:09:04.486ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:353
3992024-08-28T02:09:04.486ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:357
4002024-08-28T02:09:04.487ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:359
4012024-08-28T02:09:04.487ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key")
4022024-08-28T02:09:04.487ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key")
4032024-08-28T02:09:04.487ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:371
4042024-08-28T02:09:04.521ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
4052024-08-28T02:09:04.528ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_37700612-da7c-4ef5-9910-6b71dd793192/crypt/zone file = sled-storage/src/dataset.rs:414
4062024-08-28T02:09:04.646ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_37700612-da7c-4ef5-9910-6b71dd793192 file = sled-storage/src/dataset.rs:499
4072024-08-28T02:09:04.650ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/40b49400-5cd0-4d5c-a006-b241a1ca8261): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/probe_manager.rs:230 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261
4082024-08-28T02:09:04.655ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:440
4092024-08-28T02:09:04.655ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:357
4102024-08-28T02:09:04.655ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:342
4112024-08-28T02:09:04.655ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
4122024-08-28T02:09:04.655ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } path = /scratch/u2_8.vdev
4132024-08-28T02:09:04.663ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_8.vdev file = sled-hardware/src/disk.rs:360
4142024-08-28T02:09:04.664ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:366 id = e56ec7bf-ebda-425c-b581-1265bb74473b (zpool)
4152024-08-28T02:09:04.759ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/dataset.rs:307 zpool = ZpoolName { id: e56ec7bf-ebda-425c-b581-1265bb74473b (zpool), kind: External }
4162024-08-28T02:09:04.767ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/dataset.rs:351
4172024-08-28T02:09:04.767ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:353
4182024-08-28T02:09:04.767ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:357
4192024-08-28T02:09:04.767ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:359
4202024-08-28T02:09:04.767ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key")
4212024-08-28T02:09:04.767ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key")
4222024-08-28T02:09:04.767ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:371
4232024-08-28T02:09:04.800ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
4242024-08-28T02:09:04.807ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_e56ec7bf-ebda-425c-b581-1265bb74473b/crypt/zone file = sled-storage/src/dataset.rs:414
4252024-08-28T02:09:04.920ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_e56ec7bf-ebda-425c-b581-1265bb74473b file = sled-storage/src/dataset.rs:499
4262024-08-28T02:09:04.927ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:440
4272024-08-28T02:09:04.927ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:357
4282024-08-28T02:09:04.927ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/3dd1ecc3-6cd1-48fe-8e10-622a58ff4257/config/.omicron-physical-disks.json.tmp file = common/src/ledger.rs:199 request = omicron_physical_disks_ensure
4292024-08-28T02:09:04.927ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 3dd1ecc3-6cd1-48fe-8e10-622a58ff4257 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: fadb702a-2570-430e-a63c-e4ed78dd68d2 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 79e9b421-598b-4117-bcfb-c4abc62898c1 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: ae881951-d9c0-4515-a6b0-7e5c2297d58f (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 33490a89-0fb3-432e-bbc7-3c605249772c (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 9b46afde-44d9-45f0-a9fc-f4fa353c3113 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 5d8fb525-afb0-4464-ad9b-8e50ab9d715a (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 95dac2cb-734b-49b1-8778-a286d737d1fa (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: 37700612-da7c-4ef5-9910-6b71dd793192 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, slot_firmware_versions: [Some("synthetic 1.0")] } }, zpool_name: ZpoolName { id: e56ec7bf-ebda-425c-b581-1265bb74473b (zpool), kind: External } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
4302024-08-28T02:09:04.927ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
4312024-08-28T02:09:04.927ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
4322024-08-28T02:09:04.927ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/fadb702a-2570-430e-a63c-e4ed78dd68d2/config/.omicron-physical-disks.json.tmp file = common/src/ledger.rs:199 request = omicron_physical_disks_ensure
4332024-08-28T02:09:04.927ZINFOSledAgent: physical disks ensure: Updated storage file = sled-agent/src/sled_agent.rs:834 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261
4342024-08-28T02:09:04.927ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
4352024-08-28T02:09:04.927ZINFOSledAgent: physical disks ensure: Propagating new generation of disks file = sled-agent/src/sled_agent.rs:860 generation = Generation(1) sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261
4362024-08-28T02:09:04.927ZINFOSledAgent: physical disks ensure: Updated storage monitor file = sled-agent/src/sled_agent.rs:865 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261
4372024-08-28T02:09:04.927ZINFOSledAgent: physical disks ensure: Updated zone bundler file = sled-agent/src/sled_agent.rs:870 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261
4382024-08-28T02:09:04.928ZINFOSledAgent (ProbeManager): use_only_these_disks: Processing new request file = sled-agent/src/probe_manager.rs:121 gen = Generation(1) sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261
4392024-08-28T02:09:04.928ZINFOSledAgent: physical disks ensure: Updated probes file = sled-agent/src/sled_agent.rs:876 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261
4402024-08-28T02:09:04.928ZINFOSledAgent (InstanceManager): use_only_these_disks: Processing new request file = sled-agent/src/instance_manager.rs:803 gen = Generation(1)
4412024-08-28T02:09:04.928ZINFOSledAgent: physical disks ensure: Updated instances file = sled-agent/src/sled_agent.rs:881 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261
4422024-08-28T02:09:04.928ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3751784 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:51271 req_id = cda04676-3fc9-4693-b822-18efb9cf1509 response_code = 200 uri = /omicron-physical-disks
4432024-08-28T02:09:04.959ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(2), zones: [OmicronZoneConfig { id: f85a8e69-10b1-4e9c-982f-31422be2a907, underlay_address: fd00:1122:3344:1::1, filesystem_pool: Some(ZpoolName { id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 84c854ff-da18-4c2a-9d2f-7e15248c76a8, underlay_address: fd00:1122:3344:2::1, filesystem_pool: Some(ZpoolName { id: 79e9b421-598b-4117-bcfb-c4abc62898c1 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 79e9b421-598b-4117-bcfb-c4abc62898c1 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 01868911-e8e4-4a77-b105-b83672ce13b2, underlay_address: fd00:1122:3344:3::1, filesystem_pool: Some(ZpoolName { id: ae881951-d9c0-4515-a6b0-7e5c2297d58f (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ae881951-d9c0-4515-a6b0-7e5c2297d58f (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }] } file = sled-agent/src/rack_setup/service.rs:438 sled_address = [fd00:1122:3344:101::1]:12345
4442024-08-28T02:09:04.959ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:52321
4452024-08-28T02:09:04.960ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f85a8e69-10b1-4e9c-982f-31422be2a907, dataset_name: DatasetName { pool_name: ZpoolName { id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:406
4462024-08-28T02:09:04.960ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: f85a8e69-10b1-4e9c-982f-31422be2a907, dataset_name: DatasetName { pool_name: ZpoolName { id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool), kind: External }, kind: InternalDns }, responder: ... }
4472024-08-28T02:09:05.006ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 84c854ff-da18-4c2a-9d2f-7e15248c76a8, dataset_name: DatasetName { pool_name: ZpoolName { id: 79e9b421-598b-4117-bcfb-c4abc62898c1 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:406
4482024-08-28T02:09:05.006ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 84c854ff-da18-4c2a-9d2f-7e15248c76a8, dataset_name: DatasetName { pool_name: ZpoolName { id: 79e9b421-598b-4117-bcfb-c4abc62898c1 (zpool), kind: External }, kind: InternalDns }, responder: ... }
4492024-08-28T02:09:05.050ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 01868911-e8e4-4a77-b105-b83672ce13b2, dataset_name: DatasetName { pool_name: ZpoolName { id: ae881951-d9c0-4515-a6b0-7e5c2297d58f (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:406
4502024-08-28T02:09:05.050ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 01868911-e8e4-4a77-b105-b83672ce13b2, dataset_name: DatasetName { pool_name: ZpoolName { id: ae881951-d9c0-4515-a6b0-7e5c2297d58f (zpool), kind: External }, kind: InternalDns }, responder: ... }
4512024-08-28T02:09:05.095ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
4522024-08-28T02:09:05.095ZINFOSledAgent (ServiceManager): No ledger in /pool/int/3dd1ecc3-6cd1-48fe-8e10-622a58ff4257/config/omicron-zones.json file = common/src/ledger.rs:188
4532024-08-28T02:09:05.095ZINFOSledAgent (ServiceManager): No ledger in /pool/int/fadb702a-2570-430e-a63c-e4ed78dd68d2/config/omicron-zones.json file = common/src/ledger.rs:188
4542024-08-28T02:09:05.095ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
4552024-08-28T02:09:05.139ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
4562024-08-28T02:09:05.139ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
4572024-08-28T02:09:05.139ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
4582024-08-28T02:09:05.183ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_f85a8e69-10b1-4e9c-982f-31422be2a907 file = illumos-utils/src/zone.rs:316
4592024-08-28T02:09:05.187ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_84c854ff-da18-4c2a-9d2f-7e15248c76a8 file = illumos-utils/src/zone.rs:316
4602024-08-28T02:09:05.190ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_01868911-e8e4-4a77-b105-b83672ce13b2 file = illumos-utils/src/zone.rs:316
4612024-08-28T02:09:05.203ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_f85a8e69-10b1-4e9c-982f-31422be2a907 file = illumos-utils/src/zone.rs:354
4622024-08-28T02:09:05.207ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_84c854ff-da18-4c2a-9d2f-7e15248c76a8 file = illumos-utils/src/zone.rs:354
4632024-08-28T02:09:05.211ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_01868911-e8e4-4a77-b105-b83672ce13b2 file = illumos-utils/src/zone.rs:354
4642024-08-28T02:09:05.653ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/40b49400-5cd0-4d5c-a006-b241a1ca8261): error trying to connect: dns error: proto error: io error: No route to host (os error 148) file = sled-agent/src/probe_manager.rs:230 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261
4652024-08-28T02:09:06.746ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_01868911-e8e4-4a77-b105-b83672ce13b2: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService3'/> <propval type="astring" name="gateway" value='fd00:1122:3344:3::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:3::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:3::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:3::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
4662024-08-28T02:09:06.747ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:3::, width: 64 }
4672024-08-28T02:09:06.747ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_01868911-e8e4-4a77-b105-b83672ce13b2 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_01868911-e8e4-4a77-b105-b83672ce13b2
4682024-08-28T02:09:07.229ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_f85a8e69-10b1-4e9c-982f-31422be2a907: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService1'/> <propval type="astring" name="gateway" value='fd00:1122:3344:1::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:1::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:1::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:1::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
4692024-08-28T02:09:07.710ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:1::, width: 64 }
4702024-08-28T02:09:07.710ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_84c854ff-da18-4c2a-9d2f-7e15248c76a8: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService2'/> <propval type="astring" name="gateway" value='fd00:1122:3344:2::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:2::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:2::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:2::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
4712024-08-28T02:09:07.717ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_01868911-e8e4-4a77-b105-b83672ce13b2") failed: Property not found. retry in 62.948354ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_01868911-e8e4-4a77-b105-b83672ce13b2
4722024-08-28T02:09:07.717ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:2::, width: 64 }
4732024-08-28T02:09:07.717ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_f85a8e69-10b1-4e9c-982f-31422be2a907 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_f85a8e69-10b1-4e9c-982f-31422be2a907
4742024-08-28T02:09:07.722ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_84c854ff-da18-4c2a-9d2f-7e15248c76a8 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_84c854ff-da18-4c2a-9d2f-7e15248c76a8
4752024-08-28T02:09:07.792ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_01868911-e8e4-4a77-b105-b83672ce13b2") failed: Property not found. retry in 118.107666ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_01868911-e8e4-4a77-b105-b83672ce13b2
4762024-08-28T02:09:07.851ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_f85a8e69-10b1-4e9c-982f-31422be2a907") failed: Property not found. retry in 39.836694ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_f85a8e69-10b1-4e9c-982f-31422be2a907
4772024-08-28T02:09:07.857ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_84c854ff-da18-4c2a-9d2f-7e15248c76a8") failed: Property not found. retry in 29.510574ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_84c854ff-da18-4c2a-9d2f-7e15248c76a8
4782024-08-28T02:09:07.894ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_84c854ff-da18-4c2a-9d2f-7e15248c76a8") failed: Property not found. retry in 69.03413ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_84c854ff-da18-4c2a-9d2f-7e15248c76a8
4792024-08-28T02:09:07.901ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_f85a8e69-10b1-4e9c-982f-31422be2a907") failed: Property not found. retry in 112.864512ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_f85a8e69-10b1-4e9c-982f-31422be2a907
4802024-08-28T02:09:08.018ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_01868911-e8e4-4a77-b105-b83672ce13b2") failed: Property not found. retry in 226.474604ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_01868911-e8e4-4a77-b105-b83672ce13b2
4812024-08-28T02:09:08.071ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_f85a8e69-10b1-4e9c-982f-31422be2a907") failed: Property not found. retry in 101.992104ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_f85a8e69-10b1-4e9c-982f-31422be2a907
4822024-08-28T02:09:08.179ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_84c854ff-da18-4c2a-9d2f-7e15248c76a8") failed: Property not found. retry in 108.595757ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_84c854ff-da18-4c2a-9d2f-7e15248c76a8
4832024-08-28T02:09:08.213ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_f85a8e69-10b1-4e9c-982f-31422be2a907") failed: Property not found. retry in 494.058424ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_f85a8e69-10b1-4e9c-982f-31422be2a907
4842024-08-28T02:09:08.261ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_01868911-e8e4-4a77-b105-b83672ce13b2") failed: Property not found. retry in 306.232507ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_01868911-e8e4-4a77-b105-b83672ce13b2
4852024-08-28T02:09:08.322ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_84c854ff-da18-4c2a-9d2f-7e15248c76a8") failed: Property not found. retry in 542.376541ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_84c854ff-da18-4c2a-9d2f-7e15248c76a8
4862024-08-28T02:09:08.578ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_01868911-e8e4-4a77-b105-b83672ce13b2") failed: Property not found. retry in 723.450169ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_01868911-e8e4-4a77-b105-b83672ce13b2
4872024-08-28T02:09:08.718ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_f85a8e69-10b1-4e9c-982f-31422be2a907") failed: Property not found. retry in 477.568833ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_f85a8e69-10b1-4e9c-982f-31422be2a907
4882024-08-28T02:09:08.873ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_84c854ff-da18-4c2a-9d2f-7e15248c76a8") failed: Property not found. retry in 908.898131ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_84c854ff-da18-4c2a-9d2f-7e15248c76a8
4892024-08-28T02:09:09.205ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_f85a8e69-10b1-4e9c-982f-31422be2a907") failed: Property not found. retry in 1.409372788s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_f85a8e69-10b1-4e9c-982f-31422be2a907
4902024-08-28T02:09:09.312ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_01868911-e8e4-4a77-b105-b83672ce13b2") failed: Property not found. retry in 925.615334ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_01868911-e8e4-4a77-b105-b83672ce13b2
4912024-08-28T02:09:09.792ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_84c854ff-da18-4c2a-9d2f-7e15248c76a8") failed: Property not found. retry in 821.756356ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_84c854ff-da18-4c2a-9d2f-7e15248c76a8
4922024-08-28T02:09:10.247ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_01868911-e8e4-4a77-b105-b83672ce13b2") failed: Property not found. retry in 1.461910348s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_01868911-e8e4-4a77-b105-b83672ce13b2
4932024-08-28T02:09:10.622ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_84c854ff-da18-4c2a-9d2f-7e15248c76a8") failed: Property not found. retry in 1.266573285s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_84c854ff-da18-4c2a-9d2f-7e15248c76a8
4942024-08-28T02:09:10.629ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_f85a8e69-10b1-4e9c-982f-31422be2a907") failed: Property not found. retry in 1.17480473s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_f85a8e69-10b1-4e9c-982f-31422be2a907
4952024-08-28T02:09:10.883ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::2 exists - OK file = sled-agent/src/services.rs:4206
4962024-08-28T02:09:10.883ZINFOSledAgent (ServiceManager): Ensuring there is a default route file = sled-agent/src/services.rs:4216 gateway = fd00:1122:3344:101::1
4972024-08-28T02:09:10.889ZINFOSledAgent (ServiceManager): configuring dendrite service file = sled-agent/src/services.rs:4295
4982024-08-28T02:09:11.013ZINFOSledAgent (ServiceManager): refreshed dendrite service with new configuration file = sled-agent/src/services.rs:4343
4992024-08-28T02:09:11.013ZINFOSledAgent (ServiceManager): configuring lldp service file = sled-agent/src/services.rs:4370
5002024-08-28T02:09:11.052ZINFOSledAgent (ServiceManager): refreshed lldpd service with new configuration file = sled-agent/src/services.rs:4383
5012024-08-28T02:09:11.052ZINFOSledAgent (ServiceManager): configuring MGS service file = sled-agent/src/services.rs:4247
5022024-08-28T02:09:11.106ZINFOSledAgent (ServiceManager): refreshed MGS service with new configuration file = sled-agent/src/services.rs:4289
5032024-08-28T02:09:11.106ZINFOSledAgent (ServiceManager): configuring wicketd file = sled-agent/src/services.rs:4350 rack_subnet = fd00:1122:3344::
5042024-08-28T02:09:11.126ZINFOSledAgent (ServiceManager): refreshed wicketd service with new configuration file = sled-agent/src/services.rs:4361
5052024-08-28T02:09:11.126ZINFOSledAgent (ServiceManager): configuring mgd service file = sled-agent/src/services.rs:4403
5062024-08-28T02:09:11.212ZINFOSledAgent (ServiceManager): refreshed mgd service with new configuration file = sled-agent/src/services.rs:4435
5072024-08-28T02:09:11.213ZINFOSledAgent (ServiceManager): configuring mg-ddm service file = sled-agent/src/services.rs:4441
5082024-08-28T02:09:11.319ZINFOSledAgent (ServiceManager): refreshed mg-ddm service with new configuration file = sled-agent/src/services.rs:4482
5092024-08-28T02:09:11.319ZINFOSledAgent (ServiceManager): Determining physical location of our switch zone at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:366
5102024-08-28T02:09:11.352ZWARNSledAgent (ServiceManager): Failed to get switch ID from MGS (retrying in 66.09788ms) error = Communication Error: error sending request for url (http://[fd00:1122:3344:101::2]:12225/local/switch-id): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/bootstrap/early_networking.rs:385
5112024-08-28T02:09:11.418ZWARNSledAgent (ServiceManager): Failed to get switch ID from MGS (retrying in 70.962101ms) error = Communication Error: error sending request for url (http://[fd00:1122:3344:101::2]:12225/local/switch-id): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/bootstrap/early_networking.rs:385
5122024-08-28T02:09:11.492ZINFOSledAgent (ServiceManager): Initializing 1 Uplinks on Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:414
5132024-08-28T02:09:11.503ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::2]:49057
5142024-08-28T02:09:11.504ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 79 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::2]:49057 req_id = 87617994-056e-4b38-a735-c6ecc0b0665f response_code = 200 uri = /sled-identifiers
5152024-08-28T02:09:11.524ZINFOSledAgent (ServiceManager): Building Port Configuration file = sled-agent/src/bootstrap/early_networking.rs:676
5162024-08-28T02:09:11.524ZINFOSledAgent (ServiceManager): Checking dendrite uptime file = sled-agent/src/bootstrap/early_networking.rs:714
5172024-08-28T02:09:11.525ZINFOSledAgent (ServiceManager): Dendrite online file = sled-agent/src/bootstrap/early_networking.rs:717 uptime = 0
5182024-08-28T02:09:11.525ZINFOSledAgent (ServiceManager): Configuring default uplink on switch config = PortSettings {\n links: {\n "0": LinkSettings {\n addrs: [\n 10.151.2.174,\n ],\n params: LinkCreate {\n autoneg: false,\n fec: None,\n kr: false,\n lane: Some(\n LinkId(\n 0,\n ),\n ),\n speed: Speed40G,\n },\n },\n },\n} file = sled-agent/src/bootstrap/early_networking.rs:436
5192024-08-28T02:09:11.561ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
5202024-08-28T02:09:11.612ZINFOSledAgent (StorageManager): Received DetectedRawDisksChanged { raw_disks: {}, tx: ... } file = sled-storage/src/manager.rs:406
5212024-08-28T02:09:11.612ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
5222024-08-28T02:09:11.612ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
5232024-08-28T02:09:11.612ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
5242024-08-28T02:09:11.612ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
5252024-08-28T02:09:11.612ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
5262024-08-28T02:09:11.612ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
5272024-08-28T02:09:11.612ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
5282024-08-28T02:09:11.612ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
5292024-08-28T02:09:11.612ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
5302024-08-28T02:09:11.612ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
5312024-08-28T02:09:11.612ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
5322024-08-28T02:09:11.612ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
5332024-08-28T02:09:11.612ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
5342024-08-28T02:09:11.612ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
5352024-08-28T02:09:11.612ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
5362024-08-28T02:09:11.612ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
5372024-08-28T02:09:11.612ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
5382024-08-28T02:09:11.612ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
5392024-08-28T02:09:11.612ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
5402024-08-28T02:09:11.612ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
5412024-08-28T02:09:11.612ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
5422024-08-28T02:09:11.612ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
5432024-08-28T02:09:11.718ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_01868911-e8e4-4a77-b105-b83672ce13b2") failed: Property not found. retry in 1.299806632s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_01868911-e8e4-4a77-b105-b83672ce13b2
5442024-08-28T02:09:11.814ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_f85a8e69-10b1-4e9c-982f-31422be2a907") failed: Property not found. retry in 1.010269058s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_f85a8e69-10b1-4e9c-982f-31422be2a907
5452024-08-28T02:09:11.898ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_84c854ff-da18-4c2a-9d2f-7e15248c76a8") failed: Property not found. retry in 999.387779ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_84c854ff-da18-4c2a-9d2f-7e15248c76a8
5462024-08-28T02:09:13.041ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_internal_dns_f85a8e69-10b1-4e9c-982f-31422be2a907
5472024-08-28T02:09:13.041ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_internal_dns_84c854ff-da18-4c2a-9d2f-7e15248c76a8
5482024-08-28T02:09:13.041ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_internal_dns_01868911-e8e4-4a77-b105-b83672ce13b2
5492024-08-28T02:09:13.041ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/3dd1ecc3-6cd1-48fe-8e10-622a58ff4257/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
5502024-08-28T02:09:13.041ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/fadb702a-2570-430e-a63c-e4ed78dd68d2/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
5512024-08-28T02:09:13.041ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8081922 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:52321 req_id = 70ba73ab-05af-4086-b1d8-f4cfd9813544 response_code = 204 uri = /omicron-zones
5522024-08-28T02:09:13.043ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:1::1]:5353 file = sled-agent/src/rack_setup/service.rs:555
5532024-08-28T02:09:13.284ZWARNSledAgent (RSS): failed to write DNS configuration (will retry in 81.460758ms) dns_config_addr = [fd00:1122:3344:1::1]:5353 error_message = Communication Error: error sending request for url (http://[fd00:1122:3344:1::1]:5353/config): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:575
5542024-08-28T02:09:13.384ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:2::1]:5353 file = sled-agent/src/rack_setup/service.rs:555
5552024-08-28T02:09:13.459ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:3::1]:5353 file = sled-agent/src/rack_setup/service.rs:555
5562024-08-28T02:09:13.512ZINFOSledAgent (RSS): Configured all DNS servers file = sled-agent/src/rack_setup/service.rs:591
5572024-08-28T02:09:13.512ZINFOSledAgent (RSS): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
5582024-08-28T02:09:13.544ZINFOSledAgent (RSS): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
5592024-08-28T02:09:13.545ZINFOSledAgent (RSS): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
5602024-08-28T02:09:13.545ZINFOSledAgent (RSS): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
5612024-08-28T02:09:13.575ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(3), zones: [OmicronZoneConfig { id: f85a8e69-10b1-4e9c-982f-31422be2a907, underlay_address: fd00:1122:3344:1::1, filesystem_pool: Some(ZpoolName { id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 84c854ff-da18-4c2a-9d2f-7e15248c76a8, underlay_address: fd00:1122:3344:2::1, filesystem_pool: Some(ZpoolName { id: 79e9b421-598b-4117-bcfb-c4abc62898c1 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 79e9b421-598b-4117-bcfb-c4abc62898c1 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 01868911-e8e4-4a77-b105-b83672ce13b2, underlay_address: fd00:1122:3344:3::1, filesystem_pool: Some(ZpoolName { id: ae881951-d9c0-4515-a6b0-7e5c2297d58f (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ae881951-d9c0-4515-a6b0-7e5c2297d58f (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 3cee8112-04bf-4498-8f76-e99d79396118, underlay_address: fd00:1122:3344:101::1b, filesystem_pool: Some(ZpoolName { id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: e10be8c3-a101-4c61-ad9b-1bc4978ae517, kind: Service { id: 3cee8112-04bf-4498-8f76-e99d79396118 }, name: Name("ntp-3cee8112-04bf-4498-8f76-e99d79396118"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 185, 5])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }] } file = sled-agent/src/rack_setup/service.rs:438 sled_address = [fd00:1122:3344:101::1]:12345
5622024-08-28T02:09:13.575ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:38185
5632024-08-28T02:09:13.575ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f85a8e69-10b1-4e9c-982f-31422be2a907, dataset_name: DatasetName { pool_name: ZpoolName { id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:406
5642024-08-28T02:09:13.575ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: f85a8e69-10b1-4e9c-982f-31422be2a907, dataset_name: DatasetName { pool_name: ZpoolName { id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool), kind: External }, kind: InternalDns }, responder: ... }
5652024-08-28T02:09:13.596ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 84c854ff-da18-4c2a-9d2f-7e15248c76a8, dataset_name: DatasetName { pool_name: ZpoolName { id: 79e9b421-598b-4117-bcfb-c4abc62898c1 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:406
5662024-08-28T02:09:13.596ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 84c854ff-da18-4c2a-9d2f-7e15248c76a8, dataset_name: DatasetName { pool_name: ZpoolName { id: 79e9b421-598b-4117-bcfb-c4abc62898c1 (zpool), kind: External }, kind: InternalDns }, responder: ... }
5672024-08-28T02:09:13.616ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 01868911-e8e4-4a77-b105-b83672ce13b2, dataset_name: DatasetName { pool_name: ZpoolName { id: ae881951-d9c0-4515-a6b0-7e5c2297d58f (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:406
5682024-08-28T02:09:13.616ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 01868911-e8e4-4a77-b105-b83672ce13b2, dataset_name: DatasetName { pool_name: ZpoolName { id: ae881951-d9c0-4515-a6b0-7e5c2297d58f (zpool), kind: External }, kind: InternalDns }, responder: ... }
5692024-08-28T02:09:13.633ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
5702024-08-28T02:09:13.633ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/3dd1ecc3-6cd1-48fe-8e10-622a58ff4257/config/omicron-zones.json file = common/src/ledger.rs:177
5712024-08-28T02:09:13.633ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/fadb702a-2570-430e-a63c-e4ed78dd68d2/config/omicron-zones.json file = common/src/ledger.rs:177
5722024-08-28T02:09:13.633ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
5732024-08-28T02:09:13.644ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
5742024-08-28T02:09:13.678ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
5752024-08-28T02:09:13.678ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
5762024-08-28T02:09:13.678ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
5772024-08-28T02:09:13.708ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte0", ip: 172.30.3.5, mac: MacAddr6([168, 64, 37, 255, 185, 5]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.3.1 } }) }
5782024-08-28T02:09:13.708ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1235 zone_type = boundary_ntp
5792024-08-28T02:09:13.709ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.169 file = clients/dpd-client/src/lib.rs:83
5802024-08-28T02:09:13.709ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.169 file = clients/dpd-client/src/lib.rs:90
5812024-08-28T02:09:13.709ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.169 file = clients/dpd-client/src/lib.rs:118
5822024-08-28T02:09:13.709ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
5832024-08-28T02:09:13.735ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_ntp_3cee8112-04bf-4498-8f76-e99d79396118 file = illumos-utils/src/zone.rs:316
5842024-08-28T02:09:13.754ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_ntp_3cee8112-04bf-4498-8f76-e99d79396118 file = illumos-utils/src/zone.rs:354
5852024-08-28T02:09:14.388ZINFOSledAgent (ServiceManager): Profile for oxz_ntp_3cee8112-04bf-4498-8f76-e99d79396118: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService4'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::1b'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/chrony-setup"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="allow" value='fd00:1122:3344:100::/56'/> <propval type="boolean" name="boundary" value='true'/> <propval type="astring" name="boundary_pool" value='boundary-ntp.control-plane.oxide.internal'/> <propval type="astring" name="server" value='0.pool.ntp.org'/> </property_group> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="1.1.1.1" /> <value_node value="9.9.9.9" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/ntp"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte0'/> <propval type="astring" name="gateway" value='172.30.3.1'/> <propval type="astring" name="ip" value='172.30.3.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
5862024-08-28T02:09:14.388ZINFOSledAgent (ServiceManager): Booting oxz_ntp_3cee8112-04bf-4498-8f76-e99d79396118 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_ntp_3cee8112-04bf-4498-8f76-e99d79396118
5872024-08-28T02:09:14.519ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_3cee8112-04bf-4498-8f76-e99d79396118") failed: Property not found. retry in 29.628292ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_3cee8112-04bf-4498-8f76-e99d79396118
5882024-08-28T02:09:14.557ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_3cee8112-04bf-4498-8f76-e99d79396118") failed: Property not found. retry in 94.804105ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_3cee8112-04bf-4498-8f76-e99d79396118
5892024-08-28T02:09:14.669ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_3cee8112-04bf-4498-8f76-e99d79396118") failed: Property not found. retry in 208.508989ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_3cee8112-04bf-4498-8f76-e99d79396118
5902024-08-28T02:09:14.889ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_3cee8112-04bf-4498-8f76-e99d79396118") failed: Property not found. retry in 504.923164ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_3cee8112-04bf-4498-8f76-e99d79396118
5912024-08-28T02:09:15.404ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_3cee8112-04bf-4498-8f76-e99d79396118") failed: Property not found. retry in 833.406444ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_3cee8112-04bf-4498-8f76-e99d79396118
5922024-08-28T02:09:16.248ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_3cee8112-04bf-4498-8f76-e99d79396118") failed: Property not found. retry in 1.063361176s file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_3cee8112-04bf-4498-8f76-e99d79396118
5932024-08-28T02:09:17.320ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_3cee8112-04bf-4498-8f76-e99d79396118") failed: Property not found. retry in 864.251897ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_3cee8112-04bf-4498-8f76-e99d79396118
5942024-08-28T02:09:18.194ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_3cee8112-04bf-4498-8f76-e99d79396118") failed: Property not found. retry in 1.275447858s file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_3cee8112-04bf-4498-8f76-e99d79396118
5952024-08-28T02:09:19.491ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_ntp_3cee8112-04bf-4498-8f76-e99d79396118
5962024-08-28T02:09:19.491ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/3dd1ecc3-6cd1-48fe-8e10-622a58ff4257/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
5972024-08-28T02:09:19.491ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/fadb702a-2570-430e-a63c-e4ed78dd68d2/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
5982024-08-28T02:09:19.492ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 5916844 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:38185 req_id = ce21ab1e-b45b-4795-b0fb-c8eb1cc8d253 response_code = 204 uri = /omicron-zones
5992024-08-28T02:09:19.494ZINFOSledAgent (RSS): Waiting for rack time synchronization file = sled-agent/src/rack_setup/service.rs:632
6002024-08-28T02:09:19.525ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6012024-08-28T02:09:19.525ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:64616
6022024-08-28T02:09:19.535ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_3cee8112-04bf-4498-8f76-e99d79396118': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3847
6032024-08-28T02:09:19.536ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:902 latency_us = 10081 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:64616 req_id = f646c0b7-76de-4753-bc8d-004e6ac6f6e0 response_code = 500 uri = /timesync
6042024-08-28T02:09:19.538ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6052024-08-28T02:09:19.654ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6062024-08-28T02:09:19.655ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:64964
6072024-08-28T02:09:23.664ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_3cee8112-04bf-4498-8f76-e99d79396118': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3847
6082024-08-28T02:09:23.664ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:902 latency_us = 4009096 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:64964 req_id = 2b21e591-601c-4c75-919d-1c386a470f17 response_code = 500 uri = /timesync
6092024-08-28T02:09:23.666ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6102024-08-28T02:09:23.761ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6112024-08-28T02:09:23.761ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:46669
6122024-08-28T02:09:23.770ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_3cee8112-04bf-4498-8f76-e99d79396118': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3847
6132024-08-28T02:09:23.770ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:902 latency_us = 8963 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:46669 req_id = 1faadcb0-bc9a-452a-96fd-5a237a1b5936 response_code = 500 uri = /timesync
6142024-08-28T02:09:23.773ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6152024-08-28T02:09:23.975ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6162024-08-28T02:09:23.975ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:59050
6172024-08-28T02:09:23.983ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_3cee8112-04bf-4498-8f76-e99d79396118': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3847
6182024-08-28T02:09:23.983ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:902 latency_us = 8151 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:59050 req_id = b06f857b-54f9-4052-92de-5a11e8a6c1c3 response_code = 500 uri = /timesync
6192024-08-28T02:09:23.986ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6202024-08-28T02:09:24.162ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6212024-08-28T02:09:24.163ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:64267
6222024-08-28T02:09:27.636ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 4009193 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:64267 req_id = c2af750b-74ad-4ae0-8e12-fa430a458a5f response_code = 200 uri = /timesync
6232024-08-28T02:09:27.639ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:640
6242024-08-28T02:09:27.639ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6252024-08-28T02:09:28.103ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6262024-08-28T02:09:28.103ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:57688
6272024-08-28T02:09:28.112ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 9512 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:57688 req_id = 09c367c3-5741-4e8d-843c-25945af250fb response_code = 200 uri = /timesync
6282024-08-28T02:09:28.114ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:640
6292024-08-28T02:09:28.114ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6302024-08-28T02:09:28.179ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 249.411651ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261
6312024-08-28T02:09:28.426ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6322024-08-28T02:09:28.427ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:41625
6332024-08-28T02:09:28.436ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 9031 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:41625 req_id = 54f4af3c-18ae-4583-86a2-c1df30215c27 response_code = 200 uri = /timesync
6342024-08-28T02:09:28.439ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:640
6352024-08-28T02:09:28.439ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6362024-08-28T02:09:28.863ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6372024-08-28T02:09:28.864ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:45826
6382024-08-28T02:09:28.873ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 9056 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:45826 req_id = 06339f11-53c5-4ee5-9d13-747dabeb155b response_code = 200 uri = /timesync
6392024-08-28T02:09:28.875ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1724810966.8274498, correction: -0.035897125 } file = sled-agent/src/rack_setup/service.rs:640
6402024-08-28T02:09:28.875ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6412024-08-28T02:09:29.269ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6422024-08-28T02:09:29.269ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:33587
6432024-08-28T02:09:29.279ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 9304 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:33587 req_id = 16b91123-b427-4b13-9963-d9e3fcf1c604 response_code = 200 uri = /timesync
6442024-08-28T02:09:29.282ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1724810966.8274498, correction: -0.034795552 } file = sled-agent/src/rack_setup/service.rs:640
6452024-08-28T02:09:29.282ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6462024-08-28T02:09:29.533ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6472024-08-28T02:09:29.533ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:62744
6482024-08-28T02:09:29.541ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 7912 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:62744 req_id = 89254b9c-775b-4fe3-9602-043c7840a264 response_code = 200 uri = /timesync
6492024-08-28T02:09:29.544ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1724810966.8274498, correction: -0.034082647 } file = sled-agent/src/rack_setup/service.rs:640
6502024-08-28T02:09:29.544ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6512024-08-28T02:09:30.298ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6522024-08-28T02:09:30.298ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:62312
6532024-08-28T02:09:30.306ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 7968 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:62312 req_id = 9c6d65e1-2c93-4855-a855-bedc48b36692 response_code = 200 uri = /timesync
6542024-08-28T02:09:30.310ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1724810966.8274498, correction: -0.035596527 } file = sled-agent/src/rack_setup/service.rs:640
6552024-08-28T02:09:30.310ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6562024-08-28T02:09:31.080ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6572024-08-28T02:09:31.080ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:44511
6582024-08-28T02:09:31.088ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8280 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:44511 req_id = c7f4fda7-4169-4f0d-9ed9-b71cf4c21fad response_code = 200 uri = /timesync
6592024-08-28T02:09:31.091ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1724810966.8274498, correction: -0.033154067 } file = sled-agent/src/rack_setup/service.rs:640
6602024-08-28T02:09:31.091ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6612024-08-28T02:09:31.884ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6622024-08-28T02:09:31.884ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:36506
6632024-08-28T02:09:31.891ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 7660 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:36506 req_id = 6f9a2a17-18a1-4a32-b002-0ada429c2656 response_code = 200 uri = /timesync
6642024-08-28T02:09:31.895ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1724810966.8274498, correction: -0.033514824 } file = sled-agent/src/rack_setup/service.rs:640
6652024-08-28T02:09:31.895ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:662
6662024-08-28T02:09:32.743ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:612
6672024-08-28T02:09:32.744ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:49798
6682024-08-28T02:09:32.763ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 19838 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:49798 req_id = 60612323-c4be-4a32-8104-bdcf862fb9b6 response_code = 200 uri = /timesync
6692024-08-28T02:09:32.768ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: true, ref_id: 1244816003, ip_addr: 74.50.98.131, stratum: 3, ref_time: 1724810972.0516224, correction: -0.031924762 } file = sled-agent/src/rack_setup/service.rs:640
6702024-08-28T02:09:32.768ZINFOSledAgent (RSS): Finished setting up Internal DNS and NTP file = sled-agent/src/rack_setup/service.rs:1266
6712024-08-28T02:09:32.799ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: f85a8e69-10b1-4e9c-982f-31422be2a907, underlay_address: fd00:1122:3344:1::1, filesystem_pool: Some(ZpoolName { id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 84c854ff-da18-4c2a-9d2f-7e15248c76a8, underlay_address: fd00:1122:3344:2::1, filesystem_pool: Some(ZpoolName { id: 79e9b421-598b-4117-bcfb-c4abc62898c1 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 79e9b421-598b-4117-bcfb-c4abc62898c1 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 01868911-e8e4-4a77-b105-b83672ce13b2, underlay_address: fd00:1122:3344:3::1, filesystem_pool: Some(ZpoolName { id: ae881951-d9c0-4515-a6b0-7e5c2297d58f (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ae881951-d9c0-4515-a6b0-7e5c2297d58f (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 3cee8112-04bf-4498-8f76-e99d79396118, underlay_address: fd00:1122:3344:101::1b, filesystem_pool: Some(ZpoolName { id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: e10be8c3-a101-4c61-ad9b-1bc4978ae517, kind: Service { id: 3cee8112-04bf-4498-8f76-e99d79396118 }, name: Name("ntp-3cee8112-04bf-4498-8f76-e99d79396118"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 185, 5])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 16cc6d58-fa14-4f0f-84a7-8e8b3c7e4058, underlay_address: fd00:1122:3344:101::3, filesystem_pool: Some(ZpoolName { id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool), kind: External } } } }, OmicronZoneConfig { id: e63511dd-0799-4dc3-b977-5d8bee6f0218, underlay_address: fd00:1122:3344:101::4, filesystem_pool: Some(ZpoolName { id: 79e9b421-598b-4117-bcfb-c4abc62898c1 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 79e9b421-598b-4117-bcfb-c4abc62898c1 (zpool), kind: External } } } }, OmicronZoneConfig { id: a832edf9-7b48-46c1-a309-cbe34003f95b, underlay_address: fd00:1122:3344:101::5, filesystem_pool: Some(ZpoolName { id: ae881951-d9c0-4515-a6b0-7e5c2297d58f (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ae881951-d9c0-4515-a6b0-7e5c2297d58f (zpool), kind: External } } } }, OmicronZoneConfig { id: 7f381491-8f55-4564-a4ab-ca3915037d04, underlay_address: fd00:1122:3344:101::6, filesystem_pool: Some(ZpoolName { id: 33490a89-0fb3-432e-bbc7-3c605249772c (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 33490a89-0fb3-432e-bbc7-3c605249772c (zpool), kind: External } } } }, OmicronZoneConfig { id: 5e70cd8f-bc62-4c3d-a9c3-4361a04f47a4, underlay_address: fd00:1122:3344:101::7, filesystem_pool: Some(ZpoolName { id: 9b46afde-44d9-45f0-a9fc-f4fa353c3113 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9b46afde-44d9-45f0-a9fc-f4fa353c3113 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:438 sled_address = [fd00:1122:3344:101::1]:12345
6722024-08-28T02:09:32.799ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:56087
6732024-08-28T02:09:32.800ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f85a8e69-10b1-4e9c-982f-31422be2a907, dataset_name: DatasetName { pool_name: ZpoolName { id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:406
6742024-08-28T02:09:32.800ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: f85a8e69-10b1-4e9c-982f-31422be2a907, dataset_name: DatasetName { pool_name: ZpoolName { id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool), kind: External }, kind: InternalDns }, responder: ... }
6752024-08-28T02:09:32.823ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 84c854ff-da18-4c2a-9d2f-7e15248c76a8, dataset_name: DatasetName { pool_name: ZpoolName { id: 79e9b421-598b-4117-bcfb-c4abc62898c1 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:406
6762024-08-28T02:09:32.823ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 84c854ff-da18-4c2a-9d2f-7e15248c76a8, dataset_name: DatasetName { pool_name: ZpoolName { id: 79e9b421-598b-4117-bcfb-c4abc62898c1 (zpool), kind: External }, kind: InternalDns }, responder: ... }
6772024-08-28T02:09:32.842ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 01868911-e8e4-4a77-b105-b83672ce13b2, dataset_name: DatasetName { pool_name: ZpoolName { id: ae881951-d9c0-4515-a6b0-7e5c2297d58f (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:406
6782024-08-28T02:09:32.842ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 01868911-e8e4-4a77-b105-b83672ce13b2, dataset_name: DatasetName { pool_name: ZpoolName { id: ae881951-d9c0-4515-a6b0-7e5c2297d58f (zpool), kind: External }, kind: InternalDns }, responder: ... }
6792024-08-28T02:09:32.860ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 16cc6d58-fa14-4f0f-84a7-8e8b3c7e4058, dataset_name: DatasetName { pool_name: ZpoolName { id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:406
6802024-08-28T02:09:32.860ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 16cc6d58-fa14-4f0f-84a7-8e8b3c7e4058, dataset_name: DatasetName { pool_name: ZpoolName { id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
6812024-08-28T02:09:32.914ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: e63511dd-0799-4dc3-b977-5d8bee6f0218, dataset_name: DatasetName { pool_name: ZpoolName { id: 79e9b421-598b-4117-bcfb-c4abc62898c1 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:406
6822024-08-28T02:09:32.914ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: e63511dd-0799-4dc3-b977-5d8bee6f0218, dataset_name: DatasetName { pool_name: ZpoolName { id: 79e9b421-598b-4117-bcfb-c4abc62898c1 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
6832024-08-28T02:09:32.967ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: a832edf9-7b48-46c1-a309-cbe34003f95b, dataset_name: DatasetName { pool_name: ZpoolName { id: ae881951-d9c0-4515-a6b0-7e5c2297d58f (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:406
6842024-08-28T02:09:32.967ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: a832edf9-7b48-46c1-a309-cbe34003f95b, dataset_name: DatasetName { pool_name: ZpoolName { id: ae881951-d9c0-4515-a6b0-7e5c2297d58f (zpool), kind: External }, kind: CockroachDb }, responder: ... }
6852024-08-28T02:09:33.022ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 7f381491-8f55-4564-a4ab-ca3915037d04, dataset_name: DatasetName { pool_name: ZpoolName { id: 33490a89-0fb3-432e-bbc7-3c605249772c (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:406
6862024-08-28T02:09:33.022ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 7f381491-8f55-4564-a4ab-ca3915037d04, dataset_name: DatasetName { pool_name: ZpoolName { id: 33490a89-0fb3-432e-bbc7-3c605249772c (zpool), kind: External }, kind: CockroachDb }, responder: ... }
6872024-08-28T02:09:33.079ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 5e70cd8f-bc62-4c3d-a9c3-4361a04f47a4, dataset_name: DatasetName { pool_name: ZpoolName { id: 9b46afde-44d9-45f0-a9fc-f4fa353c3113 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:406
6882024-08-28T02:09:33.079ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 5e70cd8f-bc62-4c3d-a9c3-4361a04f47a4, dataset_name: DatasetName { pool_name: ZpoolName { id: 9b46afde-44d9-45f0-a9fc-f4fa353c3113 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
6892024-08-28T02:09:33.130ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
6902024-08-28T02:09:33.130ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/3dd1ecc3-6cd1-48fe-8e10-622a58ff4257/config/omicron-zones.json file = common/src/ledger.rs:177
6912024-08-28T02:09:33.130ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/fadb702a-2570-430e-a63c-e4ed78dd68d2/config/omicron-zones.json file = common/src/ledger.rs:177
6922024-08-28T02:09:33.130ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
6932024-08-28T02:09:33.210ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
6942024-08-28T02:09:33.210ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
6952024-08-28T02:09:33.210ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
6962024-08-28T02:09:33.210ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
6972024-08-28T02:09:33.210ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
6982024-08-28T02:09:33.298ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_7f381491-8f55-4564-a4ab-ca3915037d04 file = illumos-utils/src/zone.rs:316
6992024-08-28T02:09:33.302ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_5e70cd8f-bc62-4c3d-a9c3-4361a04f47a4 file = illumos-utils/src/zone.rs:316
7002024-08-28T02:09:33.306ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_a832edf9-7b48-46c1-a309-cbe34003f95b file = illumos-utils/src/zone.rs:316
7012024-08-28T02:09:33.310ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_16cc6d58-fa14-4f0f-84a7-8e8b3c7e4058 file = illumos-utils/src/zone.rs:316
7022024-08-28T02:09:33.315ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218 file = illumos-utils/src/zone.rs:316
7032024-08-28T02:09:33.320ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_7f381491-8f55-4564-a4ab-ca3915037d04 file = illumos-utils/src/zone.rs:354
7042024-08-28T02:09:33.324ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_5e70cd8f-bc62-4c3d-a9c3-4361a04f47a4 file = illumos-utils/src/zone.rs:354
7052024-08-28T02:09:33.328ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_a832edf9-7b48-46c1-a309-cbe34003f95b file = illumos-utils/src/zone.rs:354
7062024-08-28T02:09:33.332ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_16cc6d58-fa14-4f0f-84a7-8e8b3c7e4058 file = illumos-utils/src/zone.rs:354
7072024-08-28T02:09:33.337ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218 file = illumos-utils/src/zone.rs:354
7082024-08-28T02:09:36.325ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_7f381491-8f55-4564-a4ab-ca3915037d04: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService5'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::6]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='7f381491-8f55-4564-a4ab-ca3915037d04'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::6]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::6]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:1::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
7092024-08-28T02:09:36.325ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_7f381491-8f55-4564-a4ab-ca3915037d04 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_7f381491-8f55-4564-a4ab-ca3915037d04
7102024-08-28T02:09:36.343ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_5e70cd8f-bc62-4c3d-a9c3-4361a04f47a4: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService6'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::7'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::7]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='5e70cd8f-bc62-4c3d-a9c3-4361a04f47a4'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::7]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::7]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:1::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
7112024-08-28T02:09:36.344ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_5e70cd8f-bc62-4c3d-a9c3-4361a04f47a4 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_5e70cd8f-bc62-4c3d-a9c3-4361a04f47a4
7122024-08-28T02:09:36.353ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_a832edf9-7b48-46c1-a309-cbe34003f95b: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService7'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::5]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='a832edf9-7b48-46c1-a309-cbe34003f95b'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::5]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::5]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:1::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
7132024-08-28T02:09:36.353ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_a832edf9-7b48-46c1-a309-cbe34003f95b zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_a832edf9-7b48-46c1-a309-cbe34003f95b
7142024-08-28T02:09:36.359ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService9'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::4'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::4]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='e63511dd-0799-4dc3-b977-5d8bee6f0218'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::4]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::4]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:1::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
7152024-08-28T02:09:36.359ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218
7162024-08-28T02:09:36.379ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_16cc6d58-fa14-4f0f-84a7-8e8b3c7e4058: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService8'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::3'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::3]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='16cc6d58-fa14-4f0f-84a7-8e8b3c7e4058'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::3]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::3]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:1::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
7172024-08-28T02:09:36.379ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_16cc6d58-fa14-4f0f-84a7-8e8b3c7e4058 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_16cc6d58-fa14-4f0f-84a7-8e8b3c7e4058
7182024-08-28T02:09:36.477ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7f381491-8f55-4564-a4ab-ca3915037d04") failed: Property not found. retry in 70.087684ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7f381491-8f55-4564-a4ab-ca3915037d04
7192024-08-28T02:09:36.490ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a832edf9-7b48-46c1-a309-cbe34003f95b") failed: Property not found. retry in 70.006795ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a832edf9-7b48-46c1-a309-cbe34003f95b
7202024-08-28T02:09:36.497ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218") failed: Property not found. retry in 27.545283ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218
7212024-08-28T02:09:36.504ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5e70cd8f-bc62-4c3d-a9c3-4361a04f47a4") failed: Property not found. retry in 33.585243ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5e70cd8f-bc62-4c3d-a9c3-4361a04f47a4
7222024-08-28T02:09:36.512ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_16cc6d58-fa14-4f0f-84a7-8e8b3c7e4058") failed: Property not found. retry in 57.66439ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_16cc6d58-fa14-4f0f-84a7-8e8b3c7e4058
7232024-08-28T02:09:36.533ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218") failed: Property not found. retry in 78.418318ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218
7242024-08-28T02:09:36.627ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5e70cd8f-bc62-4c3d-a9c3-4361a04f47a4") failed: Property not found. retry in 74.337016ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5e70cd8f-bc62-4c3d-a9c3-4361a04f47a4
7252024-08-28T02:09:36.761ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a832edf9-7b48-46c1-a309-cbe34003f95b") failed: Property not found. retry in 57.220735ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a832edf9-7b48-46c1-a309-cbe34003f95b
7262024-08-28T02:09:36.789ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_16cc6d58-fa14-4f0f-84a7-8e8b3c7e4058") failed: Property not found. retry in 109.512553ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_16cc6d58-fa14-4f0f-84a7-8e8b3c7e4058
7272024-08-28T02:09:36.810ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7f381491-8f55-4564-a4ab-ca3915037d04") failed: Property not found. retry in 107.048279ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7f381491-8f55-4564-a4ab-ca3915037d04
7282024-08-28T02:09:36.844ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a832edf9-7b48-46c1-a309-cbe34003f95b") failed: Property not found. retry in 224.529828ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a832edf9-7b48-46c1-a309-cbe34003f95b
7292024-08-28T02:09:36.876ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218") failed: Property not found. retry in 144.420821ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218
7302024-08-28T02:09:36.899ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5e70cd8f-bc62-4c3d-a9c3-4361a04f47a4") failed: Property not found. retry in 262.690869ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5e70cd8f-bc62-4c3d-a9c3-4361a04f47a4
7312024-08-28T02:09:36.948ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_16cc6d58-fa14-4f0f-84a7-8e8b3c7e4058") failed: Property not found. retry in 294.785621ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_16cc6d58-fa14-4f0f-84a7-8e8b3c7e4058
7322024-08-28T02:09:36.995ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7f381491-8f55-4564-a4ab-ca3915037d04") failed: Property not found. retry in 147.675529ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7f381491-8f55-4564-a4ab-ca3915037d04
7332024-08-28T02:09:37.062ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218") failed: Property not found. retry in 536.983858ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218
7342024-08-28T02:09:37.105ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a832edf9-7b48-46c1-a309-cbe34003f95b") failed: Property not found. retry in 227.032753ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a832edf9-7b48-46c1-a309-cbe34003f95b
7352024-08-28T02:09:37.163ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7f381491-8f55-4564-a4ab-ca3915037d04") failed: Property not found. retry in 524.970339ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7f381491-8f55-4564-a4ab-ca3915037d04
7362024-08-28T02:09:37.183ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5e70cd8f-bc62-4c3d-a9c3-4361a04f47a4") failed: Property not found. retry in 579.684072ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5e70cd8f-bc62-4c3d-a9c3-4361a04f47a4
7372024-08-28T02:09:37.267ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_16cc6d58-fa14-4f0f-84a7-8e8b3c7e4058") failed: Property not found. retry in 335.15605ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_16cc6d58-fa14-4f0f-84a7-8e8b3c7e4058
7382024-08-28T02:09:37.371ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a832edf9-7b48-46c1-a309-cbe34003f95b") failed: Property not found. retry in 508.393394ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a832edf9-7b48-46c1-a309-cbe34003f95b
7392024-08-28T02:09:37.660ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_16cc6d58-fa14-4f0f-84a7-8e8b3c7e4058") failed: Property not found. retry in 945.679767ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_16cc6d58-fa14-4f0f-84a7-8e8b3c7e4058
7402024-08-28T02:09:37.717ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218") failed: Property not found. retry in 632.730333ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218
7412024-08-28T02:09:37.766ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7f381491-8f55-4564-a4ab-ca3915037d04") failed: Property not found. retry in 836.314762ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7f381491-8f55-4564-a4ab-ca3915037d04
7422024-08-28T02:09:37.848ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5e70cd8f-bc62-4c3d-a9c3-4361a04f47a4") failed: Property not found. retry in 619.852667ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5e70cd8f-bc62-4c3d-a9c3-4361a04f47a4
7432024-08-28T02:09:37.989ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a832edf9-7b48-46c1-a309-cbe34003f95b") failed: Property not found. retry in 1.182637333s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a832edf9-7b48-46c1-a309-cbe34003f95b
7442024-08-28T02:09:38.408ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218") failed: Property not found. retry in 649.472877ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218
7452024-08-28T02:09:38.540ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5e70cd8f-bc62-4c3d-a9c3-4361a04f47a4") failed: Property not found. retry in 759.360685ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5e70cd8f-bc62-4c3d-a9c3-4361a04f47a4
7462024-08-28T02:09:38.736ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7f381491-8f55-4564-a4ab-ca3915037d04") failed: Property not found. retry in 1.090727958s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7f381491-8f55-4564-a4ab-ca3915037d04
7472024-08-28T02:09:38.814ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_16cc6d58-fa14-4f0f-84a7-8e8b3c7e4058") failed: Property not found. retry in 1.297523038s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_16cc6d58-fa14-4f0f-84a7-8e8b3c7e4058
7482024-08-28T02:09:39.119ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218") failed: Property not found. retry in 1.213629237s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218
7492024-08-28T02:09:39.182ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a832edf9-7b48-46c1-a309-cbe34003f95b") failed: Property not found. retry in 547.546658ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a832edf9-7b48-46c1-a309-cbe34003f95b
7502024-08-28T02:09:39.309ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5e70cd8f-bc62-4c3d-a9c3-4361a04f47a4") failed: Property not found. retry in 1.03517164s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5e70cd8f-bc62-4c3d-a9c3-4361a04f47a4
7512024-08-28T02:09:39.738ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a832edf9-7b48-46c1-a309-cbe34003f95b") failed: Property not found. retry in 1.21671152s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a832edf9-7b48-46c1-a309-cbe34003f95b
7522024-08-28T02:09:39.833ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7f381491-8f55-4564-a4ab-ca3915037d04") failed: Property not found. retry in 608.288273ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7f381491-8f55-4564-a4ab-ca3915037d04
7532024-08-28T02:09:40.118ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_16cc6d58-fa14-4f0f-84a7-8e8b3c7e4058") failed: Property not found. retry in 1.164124203s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_16cc6d58-fa14-4f0f-84a7-8e8b3c7e4058
7542024-08-28T02:09:40.338ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218") failed: Property not found. retry in 1.181721278s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218
7552024-08-28T02:09:40.351ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5e70cd8f-bc62-4c3d-a9c3-4361a04f47a4") failed: Property not found. retry in 610.55336ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5e70cd8f-bc62-4c3d-a9c3-4361a04f47a4
7562024-08-28T02:09:40.449ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7f381491-8f55-4564-a4ab-ca3915037d04") failed: Property not found. retry in 627.353643ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7f381491-8f55-4564-a4ab-ca3915037d04
7572024-08-28T02:09:40.759ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:35869
7582024-08-28T02:09:40.759ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 89 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:35869 req_id = 5ecb4a94-d5f8-4a61-8747-5f0ff1373b4b response_code = 200 uri = /baseboard
7592024-08-28T02:09:40.961ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a832edf9-7b48-46c1-a309-cbe34003f95b") failed: Property not found. retry in 785.460206ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a832edf9-7b48-46c1-a309-cbe34003f95b
7602024-08-28T02:09:40.969ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5e70cd8f-bc62-4c3d-a9c3-4361a04f47a4") failed: Property not found. retry in 1.285969042s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5e70cd8f-bc62-4c3d-a9c3-4361a04f47a4
7612024-08-28T02:09:41.085ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7f381491-8f55-4564-a4ab-ca3915037d04") failed: Property not found. retry in 554.588053ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7f381491-8f55-4564-a4ab-ca3915037d04
7622024-08-28T02:09:41.290ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_16cc6d58-fa14-4f0f-84a7-8e8b3c7e4058") failed: Property not found. retry in 732.965619ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_16cc6d58-fa14-4f0f-84a7-8e8b3c7e4058
7632024-08-28T02:09:41.529ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218") failed: Property not found. retry in 973.354457ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218
7642024-08-28T02:09:41.651ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7f381491-8f55-4564-a4ab-ca3915037d04") failed: Property not found. retry in 1.043746127s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7f381491-8f55-4564-a4ab-ca3915037d04
7652024-08-28T02:09:41.758ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a832edf9-7b48-46c1-a309-cbe34003f95b") failed: Property not found. retry in 895.385354ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a832edf9-7b48-46c1-a309-cbe34003f95b
7662024-08-28T02:09:42.033ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_16cc6d58-fa14-4f0f-84a7-8e8b3c7e4058") failed: Property not found. retry in 1.123795225s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_16cc6d58-fa14-4f0f-84a7-8e8b3c7e4058
7672024-08-28T02:09:42.260ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5e70cd8f-bc62-4c3d-a9c3-4361a04f47a4") failed: Property not found. retry in 899.196149ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5e70cd8f-bc62-4c3d-a9c3-4361a04f47a4
7682024-08-28T02:09:42.508ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218") failed: Property not found. retry in 800.165424ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218
7692024-08-28T02:09:42.658ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a832edf9-7b48-46c1-a309-cbe34003f95b") failed: Property not found. retry in 553.843728ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a832edf9-7b48-46c1-a309-cbe34003f95b
7702024-08-28T02:09:42.700ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7f381491-8f55-4564-a4ab-ca3915037d04") failed: Property not found. retry in 964.637922ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7f381491-8f55-4564-a4ab-ca3915037d04
7712024-08-28T02:09:43.163ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_16cc6d58-fa14-4f0f-84a7-8e8b3c7e4058") failed: Property not found. retry in 567.89058ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_16cc6d58-fa14-4f0f-84a7-8e8b3c7e4058
7722024-08-28T02:09:43.170ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5e70cd8f-bc62-4c3d-a9c3-4361a04f47a4") failed: Property not found. retry in 542.415511ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5e70cd8f-bc62-4c3d-a9c3-4361a04f47a4
7732024-08-28T02:09:43.219ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a832edf9-7b48-46c1-a309-cbe34003f95b") failed: Property not found. retry in 822.100352ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a832edf9-7b48-46c1-a309-cbe34003f95b
7742024-08-28T02:09:43.316ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218") failed: Property not found. retry in 1.244471594s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218
7752024-08-28T02:09:43.430ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 427.636087ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261
7762024-08-28T02:09:43.672ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7f381491-8f55-4564-a4ab-ca3915037d04") failed: Property not found. retry in 929.926763ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7f381491-8f55-4564-a4ab-ca3915037d04
7772024-08-28T02:09:43.721ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5e70cd8f-bc62-4c3d-a9c3-4361a04f47a4") failed: Property not found. retry in 1.030733978s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5e70cd8f-bc62-4c3d-a9c3-4361a04f47a4
7782024-08-28T02:09:43.739ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_16cc6d58-fa14-4f0f-84a7-8e8b3c7e4058") failed: Property not found. retry in 565.199405ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_16cc6d58-fa14-4f0f-84a7-8e8b3c7e4058
7792024-08-28T02:09:44.049ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a832edf9-7b48-46c1-a309-cbe34003f95b") failed: Property not found. retry in 525.34166ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a832edf9-7b48-46c1-a309-cbe34003f95b
7802024-08-28T02:09:44.312ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_16cc6d58-fa14-4f0f-84a7-8e8b3c7e4058") failed: Property not found. retry in 584.318821ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_16cc6d58-fa14-4f0f-84a7-8e8b3c7e4058
7812024-08-28T02:09:44.567ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218") failed: Property not found. retry in 1.002467904s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218
7822024-08-28T02:09:44.581ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a832edf9-7b48-46c1-a309-cbe34003f95b") failed: Property not found. retry in 667.890527ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a832edf9-7b48-46c1-a309-cbe34003f95b
7832024-08-28T02:09:44.609ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7f381491-8f55-4564-a4ab-ca3915037d04") failed: Property not found. retry in 1.013866554s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7f381491-8f55-4564-a4ab-ca3915037d04
7842024-08-28T02:09:44.759ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5e70cd8f-bc62-4c3d-a9c3-4361a04f47a4") failed: Property not found. retry in 747.925133ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5e70cd8f-bc62-4c3d-a9c3-4361a04f47a4
7852024-08-28T02:09:44.905ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_16cc6d58-fa14-4f0f-84a7-8e8b3c7e4058") failed: Property not found. retry in 602.294126ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_16cc6d58-fa14-4f0f-84a7-8e8b3c7e4058
7862024-08-28T02:09:45.578ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218") failed: Property not found. retry in 734.129487ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218
7872024-08-28T02:09:46.322ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218") failed: Property not found. retry in 1.175747202s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218
7882024-08-28T02:09:47.508ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218") failed: Property not found. retry in 591.165321ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218
7892024-08-28T02:09:48.846ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218") failed: Property not found. retry in 1.096319589s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218
7902024-08-28T02:09:50.646ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218") failed: Property not found. retry in 971.978973ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218
7912024-08-28T02:09:51.634ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218") failed: Property not found. retry in 824.609497ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218
7922024-08-28T02:09:52.470ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218") failed: Property not found. retry in 1.084289206s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218
7932024-08-28T02:09:53.565ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218") failed: Property not found. retry in 981.577112ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218
7942024-08-28T02:09:54.558ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218") failed: Property not found. retry in 929.285554ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218
7952024-08-28T02:09:55.497ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218") failed: Property not found. retry in 871.608359ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218
7962024-08-28T02:09:56.393ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_cockroachdb_a832edf9-7b48-46c1-a309-cbe34003f95b
7972024-08-28T02:09:56.393ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_cockroachdb_16cc6d58-fa14-4f0f-84a7-8e8b3c7e4058
7982024-08-28T02:09:56.393ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_cockroachdb_e63511dd-0799-4dc3-b977-5d8bee6f0218
7992024-08-28T02:09:56.393ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_cockroachdb_7f381491-8f55-4564-a4ab-ca3915037d04
8002024-08-28T02:09:56.393ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_cockroachdb_5e70cd8f-bc62-4c3d-a9c3-4361a04f47a4
8012024-08-28T02:09:56.393ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/3dd1ecc3-6cd1-48fe-8e10-622a58ff4257/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
8022024-08-28T02:09:56.394ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/fadb702a-2570-430e-a63c-e4ed78dd68d2/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
8032024-08-28T02:09:56.443ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 23669427 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:56087 req_id = c66ddfae-2db0-4598-b583-56b0bd02abca response_code = 204 uri = /omicron-zones
8042024-08-28T02:09:56.483ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:53717
8052024-08-28T02:09:56.514ZINFOSledAgent (ServiceManager): Initializing CRDB Cluster - sending request to [fd00:1122:3344:101::3]:32221 file = sled-agent/src/services.rs:3681
8062024-08-28T02:09:56.913ZINFOSledAgent (ServiceManager): Formatting CRDB file = sled-agent/src/services.rs:3699
8072024-08-28T02:10:19.359ZINFOSledAgent (ServiceManager): Formatting CRDB - Completed file = sled-agent/src/services.rs:3722
8082024-08-28T02:10:19.359ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 22881217 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:53717 req_id = 4165b3aa-ee33-492c-94e9-76e39d8e51a0 response_code = 204 uri = /cockroachdb
8092024-08-28T02:10:19.360ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:39762
8102024-08-28T02:10:19.360ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 165 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:39762 req_id = b2159f76-b205-4f4e-8285-bb181fe8ff38 response_code = 200 uri = /baseboard
8112024-08-28T02:10:19.365ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 1.185553437s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261
8122024-08-28T02:10:19.396ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(5), zones: [OmicronZoneConfig { id: f85a8e69-10b1-4e9c-982f-31422be2a907, underlay_address: fd00:1122:3344:1::1, filesystem_pool: Some(ZpoolName { id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 84c854ff-da18-4c2a-9d2f-7e15248c76a8, underlay_address: fd00:1122:3344:2::1, filesystem_pool: Some(ZpoolName { id: 79e9b421-598b-4117-bcfb-c4abc62898c1 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 79e9b421-598b-4117-bcfb-c4abc62898c1 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 01868911-e8e4-4a77-b105-b83672ce13b2, underlay_address: fd00:1122:3344:3::1, filesystem_pool: Some(ZpoolName { id: ae881951-d9c0-4515-a6b0-7e5c2297d58f (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ae881951-d9c0-4515-a6b0-7e5c2297d58f (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 3cee8112-04bf-4498-8f76-e99d79396118, underlay_address: fd00:1122:3344:101::1b, filesystem_pool: Some(ZpoolName { id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: e10be8c3-a101-4c61-ad9b-1bc4978ae517, kind: Service { id: 3cee8112-04bf-4498-8f76-e99d79396118 }, name: Name("ntp-3cee8112-04bf-4498-8f76-e99d79396118"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 185, 5])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 16cc6d58-fa14-4f0f-84a7-8e8b3c7e4058, underlay_address: fd00:1122:3344:101::3, filesystem_pool: Some(ZpoolName { id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool), kind: External } } } }, OmicronZoneConfig { id: e63511dd-0799-4dc3-b977-5d8bee6f0218, underlay_address: fd00:1122:3344:101::4, filesystem_pool: Some(ZpoolName { id: 79e9b421-598b-4117-bcfb-c4abc62898c1 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 79e9b421-598b-4117-bcfb-c4abc62898c1 (zpool), kind: External } } } }, OmicronZoneConfig { id: a832edf9-7b48-46c1-a309-cbe34003f95b, underlay_address: fd00:1122:3344:101::5, filesystem_pool: Some(ZpoolName { id: ae881951-d9c0-4515-a6b0-7e5c2297d58f (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ae881951-d9c0-4515-a6b0-7e5c2297d58f (zpool), kind: External } } } }, OmicronZoneConfig { id: 7f381491-8f55-4564-a4ab-ca3915037d04, underlay_address: fd00:1122:3344:101::6, filesystem_pool: Some(ZpoolName { id: 33490a89-0fb3-432e-bbc7-3c605249772c (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 33490a89-0fb3-432e-bbc7-3c605249772c (zpool), kind: External } } } }, OmicronZoneConfig { id: 5e70cd8f-bc62-4c3d-a9c3-4361a04f47a4, underlay_address: fd00:1122:3344:101::7, filesystem_pool: Some(ZpoolName { id: 9b46afde-44d9-45f0-a9fc-f4fa353c3113 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9b46afde-44d9-45f0-a9fc-f4fa353c3113 (zpool), kind: External } } } }, OmicronZoneConfig { id: 7a899aef-896e-41b9-8810-6c5024da5a0c, underlay_address: fd00:1122:3344:101::8, filesystem_pool: Some(ZpoolName { id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool), kind: External }), zone_type: ExternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool), kind: External } }, http_address: [fd00:1122:3344:101::8]:5353, dns_address: 10.151.2.164:53, nic: NetworkInterface { id: 67a6854e-59a4-4c1d-b17a-a4ec8e40b976, kind: Service { id: 7a899aef-896e-41b9-8810-6c5024da5a0c }, name: Name("external-dns-7a899aef-896e-41b9-8810-6c5024da5a0c"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 136, 51])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } } }, OmicronZoneConfig { id: 6926ac43-4e2e-4378-9bce-42019ec72a30, underlay_address: fd00:1122:3344:101::9, filesystem_pool: Some(ZpoolName { id: 79e9b421-598b-4117-bcfb-c4abc62898c1 (zpool), kind: External }), zone_type: ExternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 79e9b421-598b-4117-bcfb-c4abc62898c1 (zpool), kind: External } }, http_address: [fd00:1122:3344:101::9]:5353, dns_address: 10.151.2.165:53, nic: NetworkInterface { id: 16f83236-bc4a-4d30-8cf5-b8e748fe9792, kind: Service { id: 6926ac43-4e2e-4378-9bce-42019ec72a30 }, name: Name("external-dns-6926ac43-4e2e-4378-9bce-42019ec72a30"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 235, 144])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } } }, OmicronZoneConfig { id: c359bc38-954c-4b35-9322-26531287be03, underlay_address: fd00:1122:3344:101::a, filesystem_pool: Some(ZpoolName { id: 95dac2cb-734b-49b1-8778-a286d737d1fa (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::a]:12221, external_ip: 10.151.2.166, nic: NetworkInterface { id: 138b549d-922d-4976-b2e2-89111b890ac2, kind: Service { id: c359bc38-954c-4b35-9322-26531287be03 }, name: Name("nexus-c359bc38-954c-4b35-9322-26531287be03"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 179, 102])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: 50ac3e36-ea5c-4742-b23f-aa0a78804739, underlay_address: fd00:1122:3344:101::b, filesystem_pool: Some(ZpoolName { id: 95dac2cb-734b-49b1-8778-a286d737d1fa (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::b]:12221, external_ip: 10.151.2.167, nic: NetworkInterface { id: cda1bc8f-c147-42a1-98e0-40955e79f4cf, kind: Service { id: 50ac3e36-ea5c-4742-b23f-aa0a78804739 }, name: Name("nexus-50ac3e36-ea5c-4742-b23f-aa0a78804739"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 180])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: 87cde93b-53d3-4924-847b-c181020478b1, underlay_address: fd00:1122:3344:101::c, filesystem_pool: Some(ZpoolName { id: 33490a89-0fb3-432e-bbc7-3c605249772c (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::c]:12221, external_ip: 10.151.2.168, nic: NetworkInterface { id: a81dd239-4fe9-4ce3-b19d-4876383286d2, kind: Service { id: 87cde93b-53d3-4924-847b-c181020478b1 }, name: Name("nexus-87cde93b-53d3-4924-847b-c181020478b1"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 249, 85])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: 1d02a2ca-a68b-4b79-b6c3-45dfe594b841, underlay_address: fd00:1122:3344:101::d, filesystem_pool: Some(ZpoolName { id: 37700612-da7c-4ef5-9910-6b71dd793192 (zpool), kind: External }), zone_type: Oximeter { address: [fd00:1122:3344:101::d]:12223 } }, OmicronZoneConfig { id: 7d4cdbf9-0586-4115-afd4-11bd8cc4917a, underlay_address: fd00:1122:3344:101::e, filesystem_pool: Some(ZpoolName { id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool), kind: External }), zone_type: Clickhouse { address: [fd00:1122:3344:101::e]:8123, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool), kind: External } } } }, OmicronZoneConfig { id: f07808bb-11dd-4aef-b8f8-6a8eaddfe9dd, underlay_address: fd00:1122:3344:101::f, filesystem_pool: Some(ZpoolName { id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::f]:17000 } }, OmicronZoneConfig { id: a9beca1c-9fcd-4a3d-9cde-db0219285191, underlay_address: fd00:1122:3344:101::10, filesystem_pool: Some(ZpoolName { id: e56ec7bf-ebda-425c-b581-1265bb74473b (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::10]:17000 } }, OmicronZoneConfig { id: 9d29a24e-a625-4e1e-8a20-b16aadbb38f4, underlay_address: fd00:1122:3344:101::11, filesystem_pool: Some(ZpoolName { id: 5d8fb525-afb0-4464-ad9b-8e50ab9d715a (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::11]:17000 } }, OmicronZoneConfig { id: 15a449aa-64f1-460c-bd5b-f4eb65dd01a3, underlay_address: fd00:1122:3344:101::12, filesystem_pool: Some(ZpoolName { id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::12]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool), kind: External } } } }, OmicronZoneConfig { id: c54505c9-f5cb-419b-8571-5173c8d0bb97, underlay_address: fd00:1122:3344:101::13, filesystem_pool: Some(ZpoolName { id: 79e9b421-598b-4117-bcfb-c4abc62898c1 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::13]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 79e9b421-598b-4117-bcfb-c4abc62898c1 (zpool), kind: External } } } }, OmicronZoneConfig { id: fea45bee-76ac-4f8d-9161-d2363f2b6f04, underlay_address: fd00:1122:3344:101::14, filesystem_pool: Some(ZpoolName { id: ae881951-d9c0-4515-a6b0-7e5c2297d58f (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::14]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ae881951-d9c0-4515-a6b0-7e5c2297d58f (zpool), kind: External } } } }, OmicronZoneConfig { id: 5b222c53-9b3a-4a9c-b345-0bfa38130f06, underlay_address: fd00:1122:3344:101::15, filesystem_pool: Some(ZpoolName { id: 33490a89-0fb3-432e-bbc7-3c605249772c (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::15]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 33490a89-0fb3-432e-bbc7-3c605249772c (zpool), kind: External } } } }, OmicronZoneConfig { id: b54f97d3-e521-4f5f-b20d-89bdd6114d3f, underlay_address: fd00:1122:3344:101::16, filesystem_pool: Some(ZpoolName { id: 9b46afde-44d9-45f0-a9fc-f4fa353c3113 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::16]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9b46afde-44d9-45f0-a9fc-f4fa353c3113 (zpool), kind: External } } } }, OmicronZoneConfig { id: e475d3e7-6b31-42b3-a2d4-0f0fc3753ded, underlay_address: fd00:1122:3344:101::17, filesystem_pool: Some(ZpoolName { id: 5d8fb525-afb0-4464-ad9b-8e50ab9d715a (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::17]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 5d8fb525-afb0-4464-ad9b-8e50ab9d715a (zpool), kind: External } } } }, OmicronZoneConfig { id: a94447d4-8244-466f-894d-de67767ef978, underlay_address: fd00:1122:3344:101::18, filesystem_pool: Some(ZpoolName { id: 95dac2cb-734b-49b1-8778-a286d737d1fa (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::18]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 95dac2cb-734b-49b1-8778-a286d737d1fa (zpool), kind: External } } } }, OmicronZoneConfig { id: c1995310-e5ce-4e60-a5e1-46140ed4eeaf, underlay_address: fd00:1122:3344:101::19, filesystem_pool: Some(ZpoolName { id: 37700612-da7c-4ef5-9910-6b71dd793192 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::19]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 37700612-da7c-4ef5-9910-6b71dd793192 (zpool), kind: External } } } }, OmicronZoneConfig { id: 72467e25-1520-4e5a-8e68-1eac43721859, underlay_address: fd00:1122:3344:101::1a, filesystem_pool: Some(ZpoolName { id: e56ec7bf-ebda-425c-b581-1265bb74473b (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::1a]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: e56ec7bf-ebda-425c-b581-1265bb74473b (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:438 sled_address = [fd00:1122:3344:101::1]:12345
8132024-08-28T02:10:19.396ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:34415
8142024-08-28T02:10:19.397ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f85a8e69-10b1-4e9c-982f-31422be2a907, dataset_name: DatasetName { pool_name: ZpoolName { id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:406
8152024-08-28T02:10:19.397ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: f85a8e69-10b1-4e9c-982f-31422be2a907, dataset_name: DatasetName { pool_name: ZpoolName { id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool), kind: External }, kind: InternalDns }, responder: ... }
8162024-08-28T02:10:19.418ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 84c854ff-da18-4c2a-9d2f-7e15248c76a8, dataset_name: DatasetName { pool_name: ZpoolName { id: 79e9b421-598b-4117-bcfb-c4abc62898c1 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:406
8172024-08-28T02:10:19.419ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 84c854ff-da18-4c2a-9d2f-7e15248c76a8, dataset_name: DatasetName { pool_name: ZpoolName { id: 79e9b421-598b-4117-bcfb-c4abc62898c1 (zpool), kind: External }, kind: InternalDns }, responder: ... }
8182024-08-28T02:10:19.440ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 01868911-e8e4-4a77-b105-b83672ce13b2, dataset_name: DatasetName { pool_name: ZpoolName { id: ae881951-d9c0-4515-a6b0-7e5c2297d58f (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:406
8192024-08-28T02:10:19.440ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 01868911-e8e4-4a77-b105-b83672ce13b2, dataset_name: DatasetName { pool_name: ZpoolName { id: ae881951-d9c0-4515-a6b0-7e5c2297d58f (zpool), kind: External }, kind: InternalDns }, responder: ... }
8202024-08-28T02:10:19.458ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 16cc6d58-fa14-4f0f-84a7-8e8b3c7e4058, dataset_name: DatasetName { pool_name: ZpoolName { id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:406
8212024-08-28T02:10:19.458ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 16cc6d58-fa14-4f0f-84a7-8e8b3c7e4058, dataset_name: DatasetName { pool_name: ZpoolName { id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
8222024-08-28T02:10:19.478ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: e63511dd-0799-4dc3-b977-5d8bee6f0218, dataset_name: DatasetName { pool_name: ZpoolName { id: 79e9b421-598b-4117-bcfb-c4abc62898c1 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:406
8232024-08-28T02:10:19.478ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: e63511dd-0799-4dc3-b977-5d8bee6f0218, dataset_name: DatasetName { pool_name: ZpoolName { id: 79e9b421-598b-4117-bcfb-c4abc62898c1 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
8242024-08-28T02:10:19.496ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: a832edf9-7b48-46c1-a309-cbe34003f95b, dataset_name: DatasetName { pool_name: ZpoolName { id: ae881951-d9c0-4515-a6b0-7e5c2297d58f (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:406
8252024-08-28T02:10:19.496ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: a832edf9-7b48-46c1-a309-cbe34003f95b, dataset_name: DatasetName { pool_name: ZpoolName { id: ae881951-d9c0-4515-a6b0-7e5c2297d58f (zpool), kind: External }, kind: CockroachDb }, responder: ... }
8262024-08-28T02:10:19.515ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 7f381491-8f55-4564-a4ab-ca3915037d04, dataset_name: DatasetName { pool_name: ZpoolName { id: 33490a89-0fb3-432e-bbc7-3c605249772c (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:406
8272024-08-28T02:10:19.515ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 7f381491-8f55-4564-a4ab-ca3915037d04, dataset_name: DatasetName { pool_name: ZpoolName { id: 33490a89-0fb3-432e-bbc7-3c605249772c (zpool), kind: External }, kind: CockroachDb }, responder: ... }
8282024-08-28T02:10:19.536ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 5e70cd8f-bc62-4c3d-a9c3-4361a04f47a4, dataset_name: DatasetName { pool_name: ZpoolName { id: 9b46afde-44d9-45f0-a9fc-f4fa353c3113 (zpool), kind: External }, kind: CockroachDb }, responder: ... }) file = sled-storage/src/manager.rs:406
8292024-08-28T02:10:19.536ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 5e70cd8f-bc62-4c3d-a9c3-4361a04f47a4, dataset_name: DatasetName { pool_name: ZpoolName { id: 9b46afde-44d9-45f0-a9fc-f4fa353c3113 (zpool), kind: External }, kind: CockroachDb }, responder: ... }
8302024-08-28T02:10:19.554ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 7a899aef-896e-41b9-8810-6c5024da5a0c, dataset_name: DatasetName { pool_name: ZpoolName { id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool), kind: External }, kind: ExternalDns }, responder: ... }) file = sled-storage/src/manager.rs:406
8312024-08-28T02:10:19.554ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 7a899aef-896e-41b9-8810-6c5024da5a0c, dataset_name: DatasetName { pool_name: ZpoolName { id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool), kind: External }, kind: ExternalDns }, responder: ... }
8322024-08-28T02:10:19.734ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 6926ac43-4e2e-4378-9bce-42019ec72a30, dataset_name: DatasetName { pool_name: ZpoolName { id: 79e9b421-598b-4117-bcfb-c4abc62898c1 (zpool), kind: External }, kind: ExternalDns }, responder: ... }) file = sled-storage/src/manager.rs:406
8332024-08-28T02:10:19.734ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 6926ac43-4e2e-4378-9bce-42019ec72a30, dataset_name: DatasetName { pool_name: ZpoolName { id: 79e9b421-598b-4117-bcfb-c4abc62898c1 (zpool), kind: External }, kind: ExternalDns }, responder: ... }
8342024-08-28T02:10:20.048ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 7d4cdbf9-0586-4115-afd4-11bd8cc4917a, dataset_name: DatasetName { pool_name: ZpoolName { id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool), kind: External }, kind: Clickhouse }, responder: ... }) file = sled-storage/src/manager.rs:406
8352024-08-28T02:10:20.048ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 7d4cdbf9-0586-4115-afd4-11bd8cc4917a, dataset_name: DatasetName { pool_name: ZpoolName { id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool), kind: External }, kind: Clickhouse }, responder: ... }
8362024-08-28T02:10:20.244ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 15a449aa-64f1-460c-bd5b-f4eb65dd01a3, dataset_name: DatasetName { pool_name: ZpoolName { id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:406
8372024-08-28T02:10:20.244ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 15a449aa-64f1-460c-bd5b-f4eb65dd01a3, dataset_name: DatasetName { pool_name: ZpoolName { id: fe881465-8582-4eba-8c48-1f5016ef66f5 (zpool), kind: External }, kind: Crucible }, responder: ... }
8382024-08-28T02:10:20.427ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c54505c9-f5cb-419b-8571-5173c8d0bb97, dataset_name: DatasetName { pool_name: ZpoolName { id: 79e9b421-598b-4117-bcfb-c4abc62898c1 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:406
8392024-08-28T02:10:20.427ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: c54505c9-f5cb-419b-8571-5173c8d0bb97, dataset_name: DatasetName { pool_name: ZpoolName { id: 79e9b421-598b-4117-bcfb-c4abc62898c1 (zpool), kind: External }, kind: Crucible }, responder: ... }
8402024-08-28T02:10:20.600ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: fea45bee-76ac-4f8d-9161-d2363f2b6f04, dataset_name: DatasetName { pool_name: ZpoolName { id: ae881951-d9c0-4515-a6b0-7e5c2297d58f (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:406
8412024-08-28T02:10:20.600ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: fea45bee-76ac-4f8d-9161-d2363f2b6f04, dataset_name: DatasetName { pool_name: ZpoolName { id: ae881951-d9c0-4515-a6b0-7e5c2297d58f (zpool), kind: External }, kind: Crucible }, responder: ... }
8422024-08-28T02:10:20.760ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 5b222c53-9b3a-4a9c-b345-0bfa38130f06, dataset_name: DatasetName { pool_name: ZpoolName { id: 33490a89-0fb3-432e-bbc7-3c605249772c (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:406
8432024-08-28T02:10:20.760ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 5b222c53-9b3a-4a9c-b345-0bfa38130f06, dataset_name: DatasetName { pool_name: ZpoolName { id: 33490a89-0fb3-432e-bbc7-3c605249772c (zpool), kind: External }, kind: Crucible }, responder: ... }
8442024-08-28T02:10:20.993ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: b54f97d3-e521-4f5f-b20d-89bdd6114d3f, dataset_name: DatasetName { pool_name: ZpoolName { id: 9b46afde-44d9-45f0-a9fc-f4fa353c3113 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:406
8452024-08-28T02:10:20.993ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: b54f97d3-e521-4f5f-b20d-89bdd6114d3f, dataset_name: DatasetName { pool_name: ZpoolName { id: 9b46afde-44d9-45f0-a9fc-f4fa353c3113 (zpool), kind: External }, kind: Crucible }, responder: ... }
8462024-08-28T02:10:21.165ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: e475d3e7-6b31-42b3-a2d4-0f0fc3753ded, dataset_name: DatasetName { pool_name: ZpoolName { id: 5d8fb525-afb0-4464-ad9b-8e50ab9d715a (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:406
8472024-08-28T02:10:21.165ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: e475d3e7-6b31-42b3-a2d4-0f0fc3753ded, dataset_name: DatasetName { pool_name: ZpoolName { id: 5d8fb525-afb0-4464-ad9b-8e50ab9d715a (zpool), kind: External }, kind: Crucible }, responder: ... }
8482024-08-28T02:10:21.387ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: a94447d4-8244-466f-894d-de67767ef978, dataset_name: DatasetName { pool_name: ZpoolName { id: 95dac2cb-734b-49b1-8778-a286d737d1fa (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:406
8492024-08-28T02:10:21.387ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: a94447d4-8244-466f-894d-de67767ef978, dataset_name: DatasetName { pool_name: ZpoolName { id: 95dac2cb-734b-49b1-8778-a286d737d1fa (zpool), kind: External }, kind: Crucible }, responder: ... }
8502024-08-28T02:10:21.610ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c1995310-e5ce-4e60-a5e1-46140ed4eeaf, dataset_name: DatasetName { pool_name: ZpoolName { id: 37700612-da7c-4ef5-9910-6b71dd793192 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:406
8512024-08-28T02:10:21.610ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: c1995310-e5ce-4e60-a5e1-46140ed4eeaf, dataset_name: DatasetName { pool_name: ZpoolName { id: 37700612-da7c-4ef5-9910-6b71dd793192 (zpool), kind: External }, kind: Crucible }, responder: ... }
8522024-08-28T02:10:21.806ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 72467e25-1520-4e5a-8e68-1eac43721859, dataset_name: DatasetName { pool_name: ZpoolName { id: e56ec7bf-ebda-425c-b581-1265bb74473b (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:406
8532024-08-28T02:10:21.806ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:773 request = NewFilesystemRequest { dataset_id: 72467e25-1520-4e5a-8e68-1eac43721859, dataset_name: DatasetName { pool_name: ZpoolName { id: e56ec7bf-ebda-425c-b581-1265bb74473b (zpool), kind: External }, kind: Crucible }, responder: ... }
8542024-08-28T02:10:22.063ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
8552024-08-28T02:10:22.063ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/3dd1ecc3-6cd1-48fe-8e10-622a58ff4257/config/omicron-zones.json file = common/src/ledger.rs:177
8562024-08-28T02:10:22.063ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/fadb702a-2570-430e-a63c-e4ed78dd68d2/config/omicron-zones.json file = common/src/ledger.rs:177
8572024-08-28T02:10:22.064ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
8582024-08-28T02:10:22.195ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
8592024-08-28T02:10:22.221ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
8602024-08-28T02:10:22.222ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
8612024-08-28T02:10:22.222ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
8622024-08-28T02:10:22.223ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
8632024-08-28T02:10:22.269ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
8642024-08-28T02:10:22.269ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
8652024-08-28T02:10:22.269ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
8662024-08-28T02:10:22.269ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
8672024-08-28T02:10:22.269ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
8682024-08-28T02:10:22.411ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
8692024-08-28T02:10:22.411ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
8702024-08-28T02:10:22.411ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
8712024-08-28T02:10:22.435ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:260
8722024-08-28T02:10:22.436ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
8732024-08-28T02:10:22.436ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
8742024-08-28T02:10:22.459ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
8752024-08-28T02:10:22.459ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
8762024-08-28T02:10:22.459ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
8772024-08-28T02:10:22.459ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3 file = illumos-utils/src/zone.rs:316
8782024-08-28T02:10:22.465ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06 file = illumos-utils/src/zone.rs:316
8792024-08-28T02:10:22.470ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f file = illumos-utils/src/zone.rs:316
8802024-08-28T02:10:22.475ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded file = illumos-utils/src/zone.rs:316
8812024-08-28T02:10:22.480ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_a94447d4-8244-466f-894d-de67767ef978 file = illumos-utils/src/zone.rs:316
8822024-08-28T02:10:22.485ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf file = illumos-utils/src/zone.rs:316
8832024-08-28T02:10:22.490ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841 file = illumos-utils/src/zone.rs:316
8842024-08-28T02:10:22.666ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
8852024-08-28T02:10:22.666ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3 file = illumos-utils/src/zone.rs:354
8862024-08-28T02:10:22.672ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06 file = illumos-utils/src/zone.rs:354
8872024-08-28T02:10:22.677ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f file = illumos-utils/src/zone.rs:354
8882024-08-28T02:10:22.681ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded file = illumos-utils/src/zone.rs:354
8892024-08-28T02:10:22.685ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_a94447d4-8244-466f-894d-de67767ef978 file = illumos-utils/src/zone.rs:354
8902024-08-28T02:10:22.690ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf file = illumos-utils/src/zone.rs:354
8912024-08-28T02:10:22.695ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841 file = illumos-utils/src/zone.rs:354
8922024-08-28T02:10:22.699ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859 file = illumos-utils/src/zone.rs:316
8932024-08-28T02:10:22.704ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4 file = illumos-utils/src/zone.rs:316
8942024-08-28T02:10:22.709ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04 file = illumos-utils/src/zone.rs:316
8952024-08-28T02:10:22.714ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191 file = illumos-utils/src/zone.rs:316
8962024-08-28T02:10:22.719ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97 file = illumos-utils/src/zone.rs:316
8972024-08-28T02:10:22.724ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_f07808bb-11dd-4aef-b8f8-6a8eaddfe9dd file = illumos-utils/src/zone.rs:316
8982024-08-28T02:10:22.729ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_clickhouse_7d4cdbf9-0586-4115-afd4-11bd8cc4917a file = illumos-utils/src/zone.rs:316
8992024-08-28T02:10:22.774ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
9002024-08-28T02:10:22.807ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
9012024-08-28T02:10:22.807ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859 file = illumos-utils/src/zone.rs:354
9022024-08-28T02:10:22.813ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4 file = illumos-utils/src/zone.rs:354
9032024-08-28T02:10:22.818ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04 file = illumos-utils/src/zone.rs:354
9042024-08-28T02:10:22.823ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191 file = illumos-utils/src/zone.rs:354
9052024-08-28T02:10:22.829ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97 file = illumos-utils/src/zone.rs:354
9062024-08-28T02:10:22.834ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_f07808bb-11dd-4aef-b8f8-6a8eaddfe9dd file = illumos-utils/src/zone.rs:354
9072024-08-28T02:10:22.841ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_clickhouse_7d4cdbf9-0586-4115-afd4-11bd8cc4917a file = illumos-utils/src/zone.rs:354
9082024-08-28T02:10:22.904ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
9092024-08-28T02:10:22.907ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
9102024-08-28T02:10:22.907ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
9112024-08-28T02:10:23.007ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte1", ip: 172.30.2.5, mac: MacAddr6([168, 64, 37, 255, 179, 102]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
9122024-08-28T02:10:23.007ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1235 zone_type = nexus
9132024-08-28T02:10:23.068ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:277
9142024-08-28T02:10:23.070ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
9152024-08-28T02:10:23.070ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
9162024-08-28T02:10:23.125ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte2", ip: 172.30.1.6, mac: MacAddr6([168, 64, 37, 255, 235, 144]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.1.1 } }) }
9172024-08-28T02:10:23.125ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1235 zone_type = external_dns
9182024-08-28T02:10:23.127ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
9192024-08-28T02:10:23.127ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
9202024-08-28T02:10:23.182ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte3", ip: 172.30.2.7, mac: MacAddr6([168, 64, 37, 255, 249, 85]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
9212024-08-28T02:10:23.182ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1235 zone_type = nexus
9222024-08-28T02:10:23.186ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
9232024-08-28T02:10:23.186ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
9242024-08-28T02:10:23.254ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte4", ip: 172.30.2.6, mac: MacAddr6([168, 64, 37, 255, 228, 180]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
9252024-08-28T02:10:23.254ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1235 zone_type = nexus
9262024-08-28T02:10:23.256ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:302
9272024-08-28T02:10:23.256ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:194
9282024-08-28T02:10:23.315ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte5", ip: 172.30.1.5, mac: MacAddr6([168, 64, 37, 255, 136, 51]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.1.1 } }) }
9292024-08-28T02:10:23.315ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1235 zone_type = external_dns
9302024-08-28T02:10:23.316ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.165 file = clients/dpd-client/src/lib.rs:83
9312024-08-28T02:10:23.316ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.165 file = clients/dpd-client/src/lib.rs:90
9322024-08-28T02:10:23.316ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.168 file = clients/dpd-client/src/lib.rs:83
9332024-08-28T02:10:23.316ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.168 file = clients/dpd-client/src/lib.rs:90
9342024-08-28T02:10:23.316ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.166 file = clients/dpd-client/src/lib.rs:83
9352024-08-28T02:10:23.316ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.166 file = clients/dpd-client/src/lib.rs:90
9362024-08-28T02:10:23.318ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.167 file = clients/dpd-client/src/lib.rs:83
9372024-08-28T02:10:23.318ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.167 file = clients/dpd-client/src/lib.rs:90
9382024-08-28T02:10:23.328ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.167 file = clients/dpd-client/src/lib.rs:118
9392024-08-28T02:10:23.330ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
9402024-08-28T02:10:23.330ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.165 file = clients/dpd-client/src/lib.rs:118
9412024-08-28T02:10:23.332ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.168 file = clients/dpd-client/src/lib.rs:118
9422024-08-28T02:10:23.334ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.166 file = clients/dpd-client/src/lib.rs:118
9432024-08-28T02:10:23.335ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
9442024-08-28T02:10:23.335ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
9452024-08-28T02:10:23.384ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
9462024-08-28T02:10:23.384ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.164 file = clients/dpd-client/src/lib.rs:83
9472024-08-28T02:10:23.384ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.164 file = clients/dpd-client/src/lib.rs:90
9482024-08-28T02:10:23.526ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_50ac3e36-ea5c-4742-b23f-aa0a78804739 file = illumos-utils/src/zone.rs:316
9492024-08-28T02:10:23.535ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.164 file = clients/dpd-client/src/lib.rs:118
9502024-08-28T02:10:23.538ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
9512024-08-28T02:10:23.538ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30 file = illumos-utils/src/zone.rs:316
9522024-08-28T02:10:23.547ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_87cde93b-53d3-4924-847b-c181020478b1 file = illumos-utils/src/zone.rs:316
9532024-08-28T02:10:23.568ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_c359bc38-954c-4b35-9322-26531287be03 file = illumos-utils/src/zone.rs:316
9542024-08-28T02:10:23.660ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30 file = illumos-utils/src/zone.rs:354
9552024-08-28T02:10:23.671ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_87cde93b-53d3-4924-847b-c181020478b1 file = illumos-utils/src/zone.rs:354
9562024-08-28T02:10:23.679ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_c359bc38-954c-4b35-9322-26531287be03 file = illumos-utils/src/zone.rs:354
9572024-08-28T02:10:23.691ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_50ac3e36-ea5c-4742-b23f-aa0a78804739 file = illumos-utils/src/zone.rs:354
9582024-08-28T02:10:23.703ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_external_dns_7a899aef-896e-41b9-8810-6c5024da5a0c file = illumos-utils/src/zone.rs:316
9592024-08-28T02:10:23.743ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_external_dns_7a899aef-896e-41b9-8810-6c5024da5a0c file = illumos-utils/src/zone.rs:354
9602024-08-28T02:10:24.820ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService18'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::11'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::11'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9612024-08-28T02:10:24.820ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4
9622024-08-28T02:10:24.900ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService20'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::10'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::10'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9632024-08-28T02:10:24.900ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191
9642024-08-28T02:10:24.917ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_a94447d4-8244-466f-894d-de67767ef978: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService14'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::18'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_95dac2cb-734b-49b1-8778-a286d737d1fa/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::18'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='35ecc937-de88-433c-93e4-d55c14ea7a31'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9652024-08-28T02:10:24.917ZINFOSledAgent (ServiceManager): Booting oxz_crucible_a94447d4-8244-466f-894d-de67767ef978 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_a94447d4-8244-466f-894d-de67767ef978
9662024-08-28T02:10:24.939ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService11'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::15'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_33490a89-0fb3-432e-bbc7-3c605249772c/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::15'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='aa61b813-41b9-47b8-8faf-0c46e9fbd6ee'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9672024-08-28T02:10:24.939ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService12'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::16'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_9b46afde-44d9-45f0-a9fc-f4fa353c3113/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::16'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='85e7bb04-adde-4546-acf3-df0c43cafc75'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9682024-08-28T02:10:24.939ZINFOSledAgent (ServiceManager): Booting oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06
9692024-08-28T02:10:24.945ZINFOSledAgent (ServiceManager): Booting oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f
9702024-08-28T02:10:24.953ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService15'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::19'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_37700612-da7c-4ef5-9910-6b71dd793192/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::19'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='0342ac45-f482-4ea0-83bd-83a859935ce2'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9712024-08-28T02:10:24.954ZINFOSledAgent (ServiceManager): Booting oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf
9722024-08-28T02:10:24.961ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService17'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::1a'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_e56ec7bf-ebda-425c-b581-1265bb74473b/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::1a'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='8815964a-40a0-4813-b31d-89872ff82775'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9732024-08-28T02:10:24.961ZINFOSledAgent (ServiceManager): Booting oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859
9742024-08-28T02:10:24.969ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService21'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::13'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_79e9b421-598b-4117-bcfb-c4abc62898c1/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::13'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='24664be4-f30f-4108-ab07-caa082fc6887'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9752024-08-28T02:10:24.969ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService19'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::14'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_ae881951-d9c0-4515-a6b0-7e5c2297d58f/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::14'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='03c6240d-c479-4c9a-9176-4cd61dd5ae13'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9762024-08-28T02:10:24.969ZINFOSledAgent (ServiceManager): Booting oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97
9772024-08-28T02:10:24.976ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService13'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::17'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_5d8fb525-afb0-4464-ad9b-8e50ab9d715a/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::17'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='7c978b51-3dc4-4a65-82ba-d36ef07a39bf'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9782024-08-28T02:10:24.976ZINFOSledAgent (ServiceManager): Booting oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04
9792024-08-28T02:10:24.981ZINFOSledAgent (ServiceManager): Booting oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded
9802024-08-28T02:10:24.994ZINFOSledAgent (ServiceManager): Profile for oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService16'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::d'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/oximeter"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="id" value='1d02a2ca-a68b-4b79-b6c3-45dfe594b841'/> <propval type="astring" name="address" value='[fd00:1122:3344:101::d]:12223'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9812024-08-28T02:10:24.995ZINFOSledAgent (ServiceManager): Booting oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841
9822024-08-28T02:10:25.021ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService10'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::12'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_fe881465-8582-4eba-8c48-1f5016ef66f5/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::12'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='fcb9aaaa-56b0-4212-82e0-1f1908d3e093'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9832024-08-28T02:10:25.021ZINFOSledAgent (ServiceManager): Booting oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3
9842024-08-28T02:10:26.103ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06") failed: Property not found. retry in 73.346172ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06
9852024-08-28T02:10:26.119ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a94447d4-8244-466f-894d-de67767ef978") failed: Property not found. retry in 64.090286ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a94447d4-8244-466f-894d-de67767ef978
9862024-08-28T02:10:26.132ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf") failed: Property not found. retry in 30.678181ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf
9872024-08-28T02:10:26.145ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841") failed: Property not found. retry in 43.390807ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841
9882024-08-28T02:10:26.160ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191") failed: Property not found. retry in 39.642711ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191
9892024-08-28T02:10:26.209ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3") failed: Property not found. retry in 37.208037ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3
9902024-08-28T02:10:26.310ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f") failed: Property not found. retry in 73.391814ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f
9912024-08-28T02:10:26.453ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded") failed: Property not found. retry in 27.910458ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded
9922024-08-28T02:10:26.506ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859") failed: Property not found. retry in 32.782837ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859
9932024-08-28T02:10:26.665ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4") failed: Property not found. retry in 44.495045ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4
9942024-08-28T02:10:30.331ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04") failed: Property not found. retry in 35.425865ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04
9952024-08-28T02:10:30.970ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97") failed: Property not found. retry in 25.060023ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97
9962024-08-28T02:10:31.243ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3") failed: Property not found. retry in 51.97115ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3
9972024-08-28T02:10:31.935ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06") failed: Property not found. retry in 108.299941ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06
9982024-08-28T02:10:32.141ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f") failed: Property not found. retry in 98.650093ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f
9992024-08-28T02:10:32.156ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded") failed: Property not found. retry in 127.597092ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded
10002024-08-28T02:10:32.170ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a94447d4-8244-466f-894d-de67767ef978") failed: Property not found. retry in 75.932865ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a94447d4-8244-466f-894d-de67767ef978
10012024-08-28T02:10:32.185ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf") failed: Property not found. retry in 116.493593ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf
10022024-08-28T02:10:32.201ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841") failed: Property not found. retry in 106.333531ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841
10032024-08-28T02:10:32.216ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859") failed: Property not found. retry in 146.274367ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859
10042024-08-28T02:10:32.229ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4") failed: Property not found. retry in 64.828732ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4
10052024-08-28T02:10:32.243ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04") failed: Property not found. retry in 55.274731ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04
10062024-08-28T02:10:32.258ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191") failed: Property not found. retry in 140.104317ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191
10072024-08-28T02:10:32.271ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97") failed: Property not found. retry in 127.258789ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97
10082024-08-28T02:10:32.288ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3") failed: Property not found. retry in 139.509593ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3
10092024-08-28T02:10:32.301ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06") failed: Property not found. retry in 241.906895ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06
10102024-08-28T02:10:32.315ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f") failed: Property not found. retry in 275.47424ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f
10112024-08-28T02:10:32.328ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded") failed: Property not found. retry in 279.475132ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded
10122024-08-28T02:10:32.341ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a94447d4-8244-466f-894d-de67767ef978") failed: Property not found. retry in 149.966751ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a94447d4-8244-466f-894d-de67767ef978
10132024-08-28T02:10:32.354ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf") failed: Property not found. retry in 157.763525ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf
10142024-08-28T02:10:32.367ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841") failed: Property not found. retry in 281.632797ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841
10152024-08-28T02:10:32.381ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859") failed: Property not found. retry in 193.574961ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859
10162024-08-28T02:10:32.395ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4") failed: Property not found. retry in 245.310085ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4
10172024-08-28T02:10:32.407ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04") failed: Property not found. retry in 292.059087ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04
10182024-08-28T02:10:32.420ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191") failed: Property not found. retry in 231.091693ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191
10192024-08-28T02:10:32.434ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97") failed: Property not found. retry in 156.556632ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97
10202024-08-28T02:10:32.448ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3") failed: Property not found. retry in 445.761652ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3
10212024-08-28T02:10:32.506ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a94447d4-8244-466f-894d-de67767ef978") failed: Property not found. retry in 390.258642ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a94447d4-8244-466f-894d-de67767ef978
10222024-08-28T02:10:32.532ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf") failed: Property not found. retry in 475.196317ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf
10232024-08-28T02:10:32.557ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06") failed: Property not found. retry in 364.278048ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06
10242024-08-28T02:10:32.588ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859") failed: Property not found. retry in 527.081128ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859
10252024-08-28T02:10:32.606ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f") failed: Property not found. retry in 242.272843ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f
10262024-08-28T02:10:32.619ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97") failed: Property not found. retry in 552.906673ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97
10272024-08-28T02:10:32.632ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded") failed: Property not found. retry in 489.048565ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded
10282024-08-28T02:10:32.655ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4") failed: Property not found. retry in 372.218448ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4
10292024-08-28T02:10:32.668ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841") failed: Property not found. retry in 385.042344ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841
10302024-08-28T02:10:32.680ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191") failed: Property not found. retry in 292.620431ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191
10312024-08-28T02:10:32.715ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04") failed: Property not found. retry in 416.913048ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04
10322024-08-28T02:10:32.864ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f") failed: Property not found. retry in 910.014974ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f
10332024-08-28T02:10:32.910ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3") failed: Property not found. retry in 899.615996ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3
10342024-08-28T02:10:32.924ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a94447d4-8244-466f-894d-de67767ef978") failed: Property not found. retry in 552.636863ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a94447d4-8244-466f-894d-de67767ef978
10352024-08-28T02:10:33.013ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06") failed: Property not found. retry in 707.433596ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06
10362024-08-28T02:10:33.087ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf") failed: Property not found. retry in 1.098761887s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf
10372024-08-28T02:10:33.110ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841") failed: Property not found. retry in 480.73222ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841
10382024-08-28T02:10:33.140ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4") failed: Property not found. retry in 1.020805402s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4
10392024-08-28T02:10:33.170ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04") failed: Property not found. retry in 982.652877ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04
10402024-08-28T02:10:33.206ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191") failed: Property not found. retry in 960.928055ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191
10412024-08-28T02:10:33.218ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97") failed: Property not found. retry in 922.811429ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97
10422024-08-28T02:10:33.231ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded") failed: Property not found. retry in 832.654098ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded
10432024-08-28T02:10:33.242ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859") failed: Property not found. retry in 553.128225ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859
10442024-08-28T02:10:33.490ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a94447d4-8244-466f-894d-de67767ef978") failed: Property not found. retry in 573.086983ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a94447d4-8244-466f-894d-de67767ef978
10452024-08-28T02:10:33.604ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841") failed: Property not found. retry in 564.895389ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841
10462024-08-28T02:10:33.732ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06") failed: Property not found. retry in 1.218842245s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06
10472024-08-28T02:10:33.788ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f") failed: Property not found. retry in 1.198895426s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f
10482024-08-28T02:10:33.807ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859") failed: Property not found. retry in 1.155491411s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859
10492024-08-28T02:10:33.820ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3") failed: Property not found. retry in 1.425832181s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3
10502024-08-28T02:10:34.075ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a94447d4-8244-466f-894d-de67767ef978") failed: Property not found. retry in 602.745687ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a94447d4-8244-466f-894d-de67767ef978
10512024-08-28T02:10:34.085ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded") failed: Property not found. retry in 1.31043675s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded
10522024-08-28T02:10:34.151ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97") failed: Property not found. retry in 825.652295ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97
10532024-08-28T02:10:34.162ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04") failed: Property not found. retry in 1.419084055s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04
10542024-08-28T02:10:34.170ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4") failed: Property not found. retry in 1.331845814s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4
10552024-08-28T02:10:34.178ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841") failed: Property not found. retry in 719.921883ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841
10562024-08-28T02:10:34.189ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191") failed: Property not found. retry in 1.199562794s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191
10572024-08-28T02:10:34.201ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf") failed: Property not found. retry in 1.115136871s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf
10582024-08-28T02:10:34.246ZINFOSledAgent (ServiceManager): Profile for oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService25'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::9'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte2'/> <propval type="astring" name="gateway" value='172.30.1.1'/> <propval type="astring" name="ip" value='172.30.1.6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/external_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::9]:5353'/> <propval type="astring" name="dns_address" value='172.30.1.6:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10592024-08-28T02:10:34.246ZINFOSledAgent (ServiceManager): Booting oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30
10602024-08-28T02:10:34.577ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30") failed: Property not found. retry in 72.389898ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30
10612024-08-28T02:10:34.662ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30") failed: Property not found. retry in 100.023248ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30
10622024-08-28T02:10:34.691ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a94447d4-8244-466f-894d-de67767ef978") failed: Property not found. retry in 1.409693383s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a94447d4-8244-466f-894d-de67767ef978
10632024-08-28T02:10:34.775ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30") failed: Property not found. retry in 130.204147ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30
10642024-08-28T02:10:34.911ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841") failed: Property not found. retry in 1.408718871s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841
10652024-08-28T02:10:34.923ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30") failed: Property not found. retry in 548.769523ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30
10662024-08-28T02:10:34.963ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06") failed: Property not found. retry in 829.668055ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06
10672024-08-28T02:10:34.974ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859") failed: Property not found. retry in 1.405645561s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859
10682024-08-28T02:10:34.989ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97") failed: Property not found. retry in 690.326991ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97
10692024-08-28T02:10:34.999ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f") failed: Property not found. retry in 828.574895ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f
10702024-08-28T02:10:35.260ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3") failed: Property not found. retry in 592.4944ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3
10712024-08-28T02:10:35.330ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf") failed: Property not found. retry in 1.445178911s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf
10722024-08-28T02:10:35.402ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191") failed: Property not found. retry in 635.752455ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191
10732024-08-28T02:10:35.411ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded") failed: Property not found. retry in 534.002823ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded
10742024-08-28T02:10:35.482ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30") failed: Property not found. retry in 958.178561ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30
10752024-08-28T02:10:35.513ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4") failed: Property not found. retry in 1.160880705s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4
10762024-08-28T02:10:35.591ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04") failed: Property not found. retry in 588.831498ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04
10772024-08-28T02:10:35.614ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 2.330926619s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261
10782024-08-28T02:10:35.690ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97") failed: Property not found. retry in 1.110726535s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97
10792024-08-28T02:10:35.758ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_87cde93b-53d3-4924-847b-c181020478b1: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService26'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::c'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte3'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.7'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10802024-08-28T02:10:35.758ZINFOSledAgent (ServiceManager): Booting oxz_nexus_87cde93b-53d3-4924-847b-c181020478b1 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_87cde93b-53d3-4924-847b-c181020478b1
10812024-08-28T02:10:35.805ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06") failed: Property not found. retry in 1.009922174s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06
10822024-08-28T02:10:35.840ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f") failed: Property not found. retry in 579.194222ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f
10832024-08-28T02:10:35.863ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3") failed: Property not found. retry in 1.121615011s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3
10842024-08-28T02:10:35.958ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded") failed: Property not found. retry in 1.02003387s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded
10852024-08-28T02:10:36.053ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191") failed: Property not found. retry in 1.312881558s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191
10862024-08-28T02:10:36.114ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a94447d4-8244-466f-894d-de67767ef978") failed: Property not found. retry in 669.166771ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a94447d4-8244-466f-894d-de67767ef978
10872024-08-28T02:10:36.194ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04") failed: Property not found. retry in 1.069888493s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04
10882024-08-28T02:10:36.331ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841") failed: Property not found. retry in 940.108773ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841
10892024-08-28T02:10:36.390ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859") failed: Property not found. retry in 1.389701856s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859
10902024-08-28T02:10:36.429ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f") failed: Property not found. retry in 1.078701615s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f
10912024-08-28T02:10:36.453ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30") failed: Property not found. retry in 898.808856ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30
10922024-08-28T02:10:36.687ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4") failed: Property not found. retry in 1.452680559s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4
10932024-08-28T02:10:36.784ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf") failed: Property not found. retry in 1.184910031s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf
10942024-08-28T02:10:36.792ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a94447d4-8244-466f-894d-de67767ef978") failed: Property not found. retry in 815.74219ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a94447d4-8244-466f-894d-de67767ef978
10952024-08-28T02:10:36.811ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97") failed: Property not found. retry in 1.300189847s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97
10962024-08-28T02:10:36.824ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06") failed: Property not found. retry in 1.285057172s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06
10972024-08-28T02:10:36.987ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded") failed: Property not found. retry in 1.245219236s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded
10982024-08-28T02:10:36.994ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3") failed: Property not found. retry in 942.383788ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3
10992024-08-28T02:10:37.272ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04") failed: Property not found. retry in 1.00813968s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04
11002024-08-28T02:10:37.280ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841") failed: Property not found. retry in 605.339925ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841
11012024-08-28T02:10:37.361ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30") failed: Property not found. retry in 1.387688175s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30
11022024-08-28T02:10:37.375ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191") failed: Property not found. retry in 539.55788ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191
11032024-08-28T02:10:37.517ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f") failed: Property not found. retry in 1.074384547s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f
11042024-08-28T02:10:37.617ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a94447d4-8244-466f-894d-de67767ef978") failed: Property not found. retry in 1.1784155s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a94447d4-8244-466f-894d-de67767ef978
11052024-08-28T02:10:37.789ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859") failed: Property not found. retry in 1.144216955s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859
11062024-08-28T02:10:37.895ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841") failed: Property not found. retry in 697.345495ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841
11072024-08-28T02:10:37.925ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191") failed: Property not found. retry in 575.720895ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191
11082024-08-28T02:10:37.946ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3") failed: Property not found. retry in 899.878389ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3
11092024-08-28T02:10:37.999ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf") failed: Property not found. retry in 1.215899809s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf
11102024-08-28T02:10:38.117ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06") failed: Property not found. retry in 749.267387ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06
11112024-08-28T02:10:38.125ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97") failed: Property not found. retry in 1.233574616s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97
11122024-08-28T02:10:38.149ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4") failed: Property not found. retry in 1.055053319s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4
11132024-08-28T02:10:38.240ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded") failed: Property not found. retry in 866.956273ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded
11142024-08-28T02:10:38.289ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04") failed: Property not found. retry in 753.652487ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04
11152024-08-28T02:10:38.510ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191") failed: Property not found. retry in 1.15725068s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191
11162024-08-28T02:10:38.601ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f") failed: Property not found. retry in 1.018184658s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f
11172024-08-28T02:10:38.609ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841") failed: Property not found. retry in 644.48688ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841
11182024-08-28T02:10:38.758ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30") failed: Property not found. retry in 539.26317ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30
11192024-08-28T02:10:38.803ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a94447d4-8244-466f-894d-de67767ef978") failed: Property not found. retry in 859.322348ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a94447d4-8244-466f-894d-de67767ef978
11202024-08-28T02:10:38.854ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3") failed: Property not found. retry in 878.10394ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3
11212024-08-28T02:10:38.876ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06") failed: Property not found. retry in 750.483437ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06
11222024-08-28T02:10:38.941ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859") failed: Property not found. retry in 1.495896273s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859
11232024-08-28T02:10:39.053ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04") failed: Property not found. retry in 698.360092ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04
11242024-08-28T02:10:39.117ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded") failed: Property not found. retry in 510.427877ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded
11252024-08-28T02:10:39.213ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4") failed: Property not found. retry in 1.199632636s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4
11262024-08-28T02:10:39.223ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf") failed: Property not found. retry in 1.303413907s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf
11272024-08-28T02:10:39.263ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841") failed: Property not found. retry in 958.253697ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841
11282024-08-28T02:10:39.307ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30") failed: Property not found. retry in 1.029222282s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30
11292024-08-28T02:10:39.368ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97") failed: Property not found. retry in 1.299984848s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97
11302024-08-28T02:10:39.628ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f") failed: Property not found. retry in 902.181757ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f
11312024-08-28T02:10:39.636ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06") failed: Property not found. retry in 1.116403191s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06
11322024-08-28T02:10:39.643ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded") failed: Property not found. retry in 777.362057ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded
11332024-08-28T02:10:39.673ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a94447d4-8244-466f-894d-de67767ef978") failed: Property not found. retry in 1.119435612s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a94447d4-8244-466f-894d-de67767ef978
11342024-08-28T02:10:39.681ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191") failed: Property not found. retry in 749.617035ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191
11352024-08-28T02:10:39.742ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3") failed: Property not found. retry in 782.629371ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3
11362024-08-28T02:10:39.760ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04") failed: Property not found. retry in 986.891649ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04
11372024-08-28T02:10:40.235ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841") failed: Property not found. retry in 561.219361ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841
11382024-08-28T02:10:40.346ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30") failed: Property not found. retry in 874.088588ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30
11392024-08-28T02:10:40.423ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4") failed: Property not found. retry in 1.236617512s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4
11402024-08-28T02:10:40.431ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded") failed: Property not found. retry in 952.586117ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded
11412024-08-28T02:10:40.440ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191") failed: Property not found. retry in 886.278491ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191
11422024-08-28T02:10:40.448ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859") failed: Property not found. retry in 962.247792ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859
11432024-08-28T02:10:40.534ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3") failed: Property not found. retry in 1.48029828s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3
11442024-08-28T02:10:40.541ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f") failed: Property not found. retry in 1.491211187s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f
11452024-08-28T02:10:40.548ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf") failed: Property not found. retry in 1.417089454s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf
11462024-08-28T02:10:40.699ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97") failed: Property not found. retry in 748.961978ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97
11472024-08-28T02:10:40.773ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04") failed: Property not found. retry in 686.878374ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04
11482024-08-28T02:10:40.788ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06") failed: Property not found. retry in 1.085290385s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06
11492024-08-28T02:10:40.808ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a94447d4-8244-466f-894d-de67767ef978") failed: Property not found. retry in 1.440985525s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a94447d4-8244-466f-894d-de67767ef978
11502024-08-28T02:10:40.821ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841") failed: Property not found. retry in 666.117752ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841
11512024-08-28T02:10:40.829ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_f07808bb-11dd-4aef-b8f8-6a8eaddfe9dd: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService22'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::f'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::f'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11522024-08-28T02:10:44.865ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30") failed: Property not found. retry in 808.816512ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30
11532024-08-28T02:10:44.865ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_f07808bb-11dd-4aef-b8f8-6a8eaddfe9dd zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_f07808bb-11dd-4aef-b8f8-6a8eaddfe9dd
11542024-08-28T02:10:44.915ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3") failed: Property not found. retry in 1.021865699s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3
11552024-08-28T02:10:44.940ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06") failed: Property not found. retry in 1.202450019s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06
11562024-08-28T02:10:44.966ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_87cde93b-53d3-4924-847b-c181020478b1") failed: Property not found. retry in 42.395838ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_87cde93b-53d3-4924-847b-c181020478b1
11572024-08-28T02:10:44.984ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f") failed: Property not found. retry in 519.150601ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f
11582024-08-28T02:10:44.995ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded") failed: Property not found. retry in 1.416979111s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded
11592024-08-28T02:10:45.004ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a94447d4-8244-466f-894d-de67767ef978") failed: Property not found. retry in 1.175977235s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a94447d4-8244-466f-894d-de67767ef978
11602024-08-28T02:10:45.015ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf") failed: Property not found. retry in 1.192787487s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf
11612024-08-28T02:10:45.024ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841") failed: Property not found. retry in 594.090049ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841
11622024-08-28T02:10:45.034ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859") failed: Property not found. retry in 871.402895ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859
11632024-08-28T02:10:45.044ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4") failed: Property not found. retry in 1.385169468s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4
11642024-08-28T02:10:45.054ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04") failed: Property not found. retry in 1.300460779s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04
11652024-08-28T02:10:45.064ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191") failed: Property not found. retry in 607.987806ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191
11662024-08-28T02:10:45.072ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97") failed: Property not found. retry in 1.405036315s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97
11672024-08-28T02:10:45.081ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_87cde93b-53d3-4924-847b-c181020478b1") failed: Property not found. retry in 105.88279ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_87cde93b-53d3-4924-847b-c181020478b1
11682024-08-28T02:10:45.083ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_c359bc38-954c-4b35-9322-26531287be03: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService27'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::a'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte1'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11692024-08-28T02:10:45.083ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_50ac3e36-ea5c-4742-b23f-aa0a78804739: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService24'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::b'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte4'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11702024-08-28T02:10:45.083ZINFOSledAgent (ServiceManager): Booting oxz_nexus_c359bc38-954c-4b35-9322-26531287be03 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_c359bc38-954c-4b35-9322-26531287be03
11712024-08-28T02:10:45.090ZINFOSledAgent (ServiceManager): Booting oxz_nexus_50ac3e36-ea5c-4742-b23f-aa0a78804739 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_50ac3e36-ea5c-4742-b23f-aa0a78804739
11722024-08-28T02:10:45.133ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_f07808bb-11dd-4aef-b8f8-6a8eaddfe9dd") failed: Property not found. retry in 71.532332ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_f07808bb-11dd-4aef-b8f8-6a8eaddfe9dd
11732024-08-28T02:10:45.200ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_87cde93b-53d3-4924-847b-c181020478b1") failed: Property not found. retry in 152.312329ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_87cde93b-53d3-4924-847b-c181020478b1
11742024-08-28T02:10:45.225ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_f07808bb-11dd-4aef-b8f8-6a8eaddfe9dd") failed: Property not found. retry in 104.155034ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_f07808bb-11dd-4aef-b8f8-6a8eaddfe9dd
11752024-08-28T02:10:45.290ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_50ac3e36-ea5c-4742-b23f-aa0a78804739") failed: Property not found. retry in 55.991335ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_50ac3e36-ea5c-4742-b23f-aa0a78804739
11762024-08-28T02:10:45.297ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_c359bc38-954c-4b35-9322-26531287be03") failed: Property not found. retry in 39.73122ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_c359bc38-954c-4b35-9322-26531287be03
11772024-08-28T02:10:45.342ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_f07808bb-11dd-4aef-b8f8-6a8eaddfe9dd") failed: Property not found. retry in 289.192862ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_f07808bb-11dd-4aef-b8f8-6a8eaddfe9dd
11782024-08-28T02:10:45.349ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_c359bc38-954c-4b35-9322-26531287be03") failed: Property not found. retry in 97.901792ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_c359bc38-954c-4b35-9322-26531287be03
11792024-08-28T02:10:45.356ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_50ac3e36-ea5c-4742-b23f-aa0a78804739") failed: Property not found. retry in 64.109155ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_50ac3e36-ea5c-4742-b23f-aa0a78804739
11802024-08-28T02:10:45.365ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_87cde93b-53d3-4924-847b-c181020478b1") failed: Property not found. retry in 350.546603ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_87cde93b-53d3-4924-847b-c181020478b1
11812024-08-28T02:10:45.405ZINFOSledAgent (ServiceManager): Profile for oxz_clickhouse_7d4cdbf9-0586-4115-afd4-11bd8cc4917a: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService23'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::e'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/clickhouse"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::e'/> <propval type="astring" name="listen_port" value='8123'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:3::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:1::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/clickhouse-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="clickhouse_address" value='[fd00:1122:3344:101::e]:8123'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::e]:8888'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11822024-08-28T02:10:45.406ZINFOSledAgent (ServiceManager): Booting oxz_clickhouse_7d4cdbf9-0586-4115-afd4-11bd8cc4917a zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_clickhouse_7d4cdbf9-0586-4115-afd4-11bd8cc4917a
11832024-08-28T02:10:45.429ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_50ac3e36-ea5c-4742-b23f-aa0a78804739") failed: Property not found. retry in 224.574732ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_50ac3e36-ea5c-4742-b23f-aa0a78804739
11842024-08-28T02:10:45.456ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_c359bc38-954c-4b35-9322-26531287be03") failed: Property not found. retry in 267.59784ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_c359bc38-954c-4b35-9322-26531287be03
11852024-08-28T02:10:45.519ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f") failed: Property not found. retry in 1.289381208s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f
11862024-08-28T02:10:45.632ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841") failed: Property not found. retry in 589.453964ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841
11872024-08-28T02:10:45.648ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_f07808bb-11dd-4aef-b8f8-6a8eaddfe9dd") failed: Property not found. retry in 515.606985ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_f07808bb-11dd-4aef-b8f8-6a8eaddfe9dd
11882024-08-28T02:10:45.666ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_50ac3e36-ea5c-4742-b23f-aa0a78804739") failed: Property not found. retry in 500.253538ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_50ac3e36-ea5c-4742-b23f-aa0a78804739
11892024-08-28T02:10:45.683ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191") failed: Property not found. retry in 551.123386ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191
11902024-08-28T02:10:45.692ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30") failed: Property not found. retry in 1.361555976s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30
11912024-08-28T02:10:45.700ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_7d4cdbf9-0586-4115-afd4-11bd8cc4917a") failed: Property not found. retry in 57.647697ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_7d4cdbf9-0586-4115-afd4-11bd8cc4917a
11922024-08-28T02:10:45.725ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_87cde93b-53d3-4924-847b-c181020478b1") failed: Property not found. retry in 446.145115ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_87cde93b-53d3-4924-847b-c181020478b1
11932024-08-28T02:10:45.737ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_c359bc38-954c-4b35-9322-26531287be03") failed: Property not found. retry in 415.845765ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_c359bc38-954c-4b35-9322-26531287be03
11942024-08-28T02:10:45.767ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_7d4cdbf9-0586-4115-afd4-11bd8cc4917a") failed: Property not found. retry in 102.219268ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_7d4cdbf9-0586-4115-afd4-11bd8cc4917a
11952024-08-28T02:10:45.865ZINFOSledAgent (ServiceManager): Profile for oxz_external_dns_7a899aef-896e-41b9-8810-6c5024da5a0c: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService28'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::8'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte5'/> <propval type="astring" name="gateway" value='172.30.1.1'/> <propval type="astring" name="ip" value='172.30.1.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/external_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::8]:5353'/> <propval type="astring" name="dns_address" value='172.30.1.5:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11962024-08-28T02:10:45.865ZINFOSledAgent (ServiceManager): Booting oxz_external_dns_7a899aef-896e-41b9-8810-6c5024da5a0c zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_external_dns_7a899aef-896e-41b9-8810-6c5024da5a0c
11972024-08-28T02:10:45.880ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_7d4cdbf9-0586-4115-afd4-11bd8cc4917a") failed: Property not found. retry in 234.970919ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_7d4cdbf9-0586-4115-afd4-11bd8cc4917a
11982024-08-28T02:10:45.917ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859") failed: Property not found. retry in 926.72904ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859
11992024-08-28T02:10:45.946ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3") failed: Property not found. retry in 709.739996ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3
12002024-08-28T02:10:46.053ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7a899aef-896e-41b9-8810-6c5024da5a0c") failed: Property not found. retry in 44.490459ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7a899aef-896e-41b9-8810-6c5024da5a0c
12012024-08-28T02:10:46.107ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7a899aef-896e-41b9-8810-6c5024da5a0c") failed: Property not found. retry in 147.871509ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7a899aef-896e-41b9-8810-6c5024da5a0c
12022024-08-28T02:10:46.125ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_7d4cdbf9-0586-4115-afd4-11bd8cc4917a") failed: Property not found. retry in 544.872323ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_7d4cdbf9-0586-4115-afd4-11bd8cc4917a
12032024-08-28T02:10:46.152ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06") failed: Property not found. retry in 1.292872792s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06
12042024-08-28T02:10:46.162ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_c359bc38-954c-4b35-9322-26531287be03") failed: Property not found. retry in 644.446149ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_c359bc38-954c-4b35-9322-26531287be03
12052024-08-28T02:10:46.174ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_f07808bb-11dd-4aef-b8f8-6a8eaddfe9dd") failed: Property not found. retry in 729.067008ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_f07808bb-11dd-4aef-b8f8-6a8eaddfe9dd
12062024-08-28T02:10:46.193ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_87cde93b-53d3-4924-847b-c181020478b1") failed: Property not found. retry in 821.349091ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_87cde93b-53d3-4924-847b-c181020478b1
12072024-08-28T02:10:46.207ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_50ac3e36-ea5c-4742-b23f-aa0a78804739") failed: Property not found. retry in 1.090108024s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_50ac3e36-ea5c-4742-b23f-aa0a78804739
12082024-08-28T02:10:46.224ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a94447d4-8244-466f-894d-de67767ef978") failed: Property not found. retry in 643.225975ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a94447d4-8244-466f-894d-de67767ef978
12092024-08-28T02:10:46.255ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf") failed: Property not found. retry in 1.063129983s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf
12102024-08-28T02:10:46.265ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841") failed: Property not found. retry in 1.411063274s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841
12112024-08-28T02:10:46.275ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191") failed: Property not found. retry in 1.239817194s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191
12122024-08-28T02:10:46.286ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7a899aef-896e-41b9-8810-6c5024da5a0c") failed: Property not found. retry in 158.507925ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7a899aef-896e-41b9-8810-6c5024da5a0c
12132024-08-28T02:10:46.366ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04") failed: Property not found. retry in 1.30877214s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04
12142024-08-28T02:10:46.424ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded") failed: Property not found. retry in 1.173632089s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded
12152024-08-28T02:10:46.439ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4") failed: Property not found. retry in 1.47164191s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4
12162024-08-28T02:10:46.454ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7a899aef-896e-41b9-8810-6c5024da5a0c") failed: Property not found. retry in 427.235482ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7a899aef-896e-41b9-8810-6c5024da5a0c
12172024-08-28T02:10:46.486ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97") failed: Property not found. retry in 1.282067612s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97
12182024-08-28T02:10:46.668ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3") failed: Property not found. retry in 1.077038988s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3
12192024-08-28T02:10:46.681ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_7d4cdbf9-0586-4115-afd4-11bd8cc4917a") failed: Property not found. retry in 803.255802ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_7d4cdbf9-0586-4115-afd4-11bd8cc4917a
12202024-08-28T02:10:46.820ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_c359bc38-954c-4b35-9322-26531287be03") failed: Property not found. retry in 1.400360132s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_c359bc38-954c-4b35-9322-26531287be03
12212024-08-28T02:10:46.830ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f") failed: Property not found. retry in 1.344930887s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f
12222024-08-28T02:10:46.854ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859") failed: Property not found. retry in 947.967216ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859
12232024-08-28T02:10:46.877ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a94447d4-8244-466f-894d-de67767ef978") failed: Property not found. retry in 1.165037214s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a94447d4-8244-466f-894d-de67767ef978
12242024-08-28T02:10:46.897ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7a899aef-896e-41b9-8810-6c5024da5a0c") failed: Property not found. retry in 558.940463ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7a899aef-896e-41b9-8810-6c5024da5a0c
12252024-08-28T02:10:46.913ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_f07808bb-11dd-4aef-b8f8-6a8eaddfe9dd") failed: Property not found. retry in 719.598393ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_f07808bb-11dd-4aef-b8f8-6a8eaddfe9dd
12262024-08-28T02:10:47.035ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_87cde93b-53d3-4924-847b-c181020478b1") failed: Property not found. retry in 822.612255ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_87cde93b-53d3-4924-847b-c181020478b1
12272024-08-28T02:10:47.074ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30") failed: Property not found. retry in 545.435566ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30
12282024-08-28T02:10:47.309ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_50ac3e36-ea5c-4742-b23f-aa0a78804739") failed: Property not found. retry in 1.341357617s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_50ac3e36-ea5c-4742-b23f-aa0a78804739
12292024-08-28T02:10:47.332ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf") failed: Property not found. retry in 1.193948287s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf
12302024-08-28T02:10:47.460ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06") failed: Property not found. retry in 906.093457ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06
12312024-08-28T02:10:47.474ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7a899aef-896e-41b9-8810-6c5024da5a0c") failed: Property not found. retry in 1.413424956s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7a899aef-896e-41b9-8810-6c5024da5a0c
12322024-08-28T02:10:47.499ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_7d4cdbf9-0586-4115-afd4-11bd8cc4917a") failed: Property not found. retry in 1.401238888s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_7d4cdbf9-0586-4115-afd4-11bd8cc4917a
12332024-08-28T02:10:47.530ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191") failed: Property not found. retry in 1.006116355s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191
12342024-08-28T02:10:47.610ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded") failed: Property not found. retry in 585.699525ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded
12352024-08-28T02:10:47.633ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30") failed: Property not found. retry in 907.640663ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30
12362024-08-28T02:10:47.645ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_f07808bb-11dd-4aef-b8f8-6a8eaddfe9dd") failed: Property not found. retry in 1.453934344s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_f07808bb-11dd-4aef-b8f8-6a8eaddfe9dd
12372024-08-28T02:10:47.690ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04") failed: Property not found. retry in 976.807463ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04
12382024-08-28T02:10:47.704ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841") failed: Property not found. retry in 821.868771ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841
12392024-08-28T02:10:47.759ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3") failed: Property not found. retry in 780.487195ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3
12402024-08-28T02:10:47.783ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97") failed: Property not found. retry in 1.219851011s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97
12412024-08-28T02:10:47.818ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859") failed: Property not found. retry in 772.261053ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859
12422024-08-28T02:10:47.878ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_87cde93b-53d3-4924-847b-c181020478b1") failed: Property not found. retry in 1.355790761s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_87cde93b-53d3-4924-847b-c181020478b1
12432024-08-28T02:10:47.928ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4") failed: Property not found. retry in 826.884609ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4
12442024-08-28T02:10:48.058ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a94447d4-8244-466f-894d-de67767ef978") failed: Property not found. retry in 804.569358ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a94447d4-8244-466f-894d-de67767ef978
12452024-08-28T02:10:48.194ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f") failed: Property not found. retry in 646.300469ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f
12462024-08-28T02:10:48.217ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded") failed: Property not found. retry in 764.42863ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded
12472024-08-28T02:10:48.242ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_c359bc38-954c-4b35-9322-26531287be03") failed: Property not found. retry in 1.212998601s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_c359bc38-954c-4b35-9322-26531287be03
12482024-08-28T02:10:48.383ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06") failed: Property not found. retry in 1.162040643s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06
12492024-08-28T02:10:48.544ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf") failed: Property not found. retry in 1.294291574s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf
12502024-08-28T02:10:48.559ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841") failed: Property not found. retry in 1.462991539s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841
12512024-08-28T02:10:48.574ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30") failed: Property not found. retry in 754.313679ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30
12522024-08-28T02:10:48.593ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3") failed: Property not found. retry in 689.328696ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3
12532024-08-28T02:10:48.610ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191") failed: Property not found. retry in 1.301516917s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191
12542024-08-28T02:10:48.626ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859") failed: Property not found. retry in 729.248534ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859
12552024-08-28T02:10:48.672ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_50ac3e36-ea5c-4742-b23f-aa0a78804739") failed: Property not found. retry in 564.76341ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_50ac3e36-ea5c-4742-b23f-aa0a78804739
12562024-08-28T02:10:48.699ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04") failed: Property not found. retry in 686.250684ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04
12572024-08-28T02:10:48.777ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4") failed: Property not found. retry in 747.191725ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4
12582024-08-28T02:10:48.858ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f") failed: Property not found. retry in 951.422779ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f
12592024-08-28T02:10:48.883ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a94447d4-8244-466f-894d-de67767ef978") failed: Property not found. retry in 967.191329ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a94447d4-8244-466f-894d-de67767ef978
12602024-08-28T02:10:48.918ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7a899aef-896e-41b9-8810-6c5024da5a0c") failed: Property not found. retry in 1.046303383s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7a899aef-896e-41b9-8810-6c5024da5a0c
12612024-08-28T02:10:48.943ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_7d4cdbf9-0586-4115-afd4-11bd8cc4917a") failed: Property not found. retry in 1.309567757s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_7d4cdbf9-0586-4115-afd4-11bd8cc4917a
12622024-08-28T02:10:49.000ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded") failed: Property not found. retry in 1.008422747s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded
12632024-08-28T02:10:49.018ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97") failed: Property not found. retry in 785.4808ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97
12642024-08-28T02:10:49.113ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_f07808bb-11dd-4aef-b8f8-6a8eaddfe9dd") failed: Property not found. retry in 1.167130807s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_f07808bb-11dd-4aef-b8f8-6a8eaddfe9dd
12652024-08-28T02:10:49.244ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_87cde93b-53d3-4924-847b-c181020478b1") failed: Property not found. retry in 747.929352ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_87cde93b-53d3-4924-847b-c181020478b1
12662024-08-28T02:10:49.251ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_50ac3e36-ea5c-4742-b23f-aa0a78804739") failed: Property not found. retry in 856.3717ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_50ac3e36-ea5c-4742-b23f-aa0a78804739
12672024-08-28T02:10:49.290ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3") failed: Property not found. retry in 731.571911ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3
12682024-08-28T02:10:49.338ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30") failed: Property not found. retry in 1.169862672s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30
12692024-08-28T02:10:49.366ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859") failed: Property not found. retry in 1.035651836s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859
12702024-08-28T02:10:49.395ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04") failed: Property not found. retry in 574.688322ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04
12712024-08-28T02:10:49.427ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:44282
12722024-08-28T02:10:49.429ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1691 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:44282 req_id = 23c939a8-043d-46d0-a4e9-8e88f53e4adf response_code = 200 uri = /baseboard
12732024-08-28T02:10:49.466ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_c359bc38-954c-4b35-9322-26531287be03") failed: Property not found. retry in 1.308343067s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_c359bc38-954c-4b35-9322-26531287be03
12742024-08-28T02:10:49.535ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4") failed: Property not found. retry in 1.282372625s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4
12752024-08-28T02:10:49.555ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06") failed: Property not found. retry in 887.897288ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06
12762024-08-28T02:10:49.814ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97") failed: Property not found. retry in 633.571904ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97
12772024-08-28T02:10:49.824ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f") failed: Property not found. retry in 838.891942ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f
12782024-08-28T02:10:49.848ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf") failed: Property not found. retry in 902.668321ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf
12792024-08-28T02:10:49.861ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a94447d4-8244-466f-894d-de67767ef978") failed: Property not found. retry in 712.741417ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a94447d4-8244-466f-894d-de67767ef978
12802024-08-28T02:10:49.920ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191") failed: Property not found. retry in 971.729529ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191
12812024-08-28T02:10:49.974ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7a899aef-896e-41b9-8810-6c5024da5a0c") failed: Property not found. retry in 724.544677ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7a899aef-896e-41b9-8810-6c5024da5a0c
12822024-08-28T02:10:49.984ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04") failed: Property not found. retry in 1.122163694s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04
12832024-08-28T02:10:50.001ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_87cde93b-53d3-4924-847b-c181020478b1") failed: Property not found. retry in 813.027751ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_87cde93b-53d3-4924-847b-c181020478b1
12842024-08-28T02:10:50.018ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded") failed: Property not found. retry in 966.749667ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded
12852024-08-28T02:10:50.030ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3") failed: Property not found. retry in 1.293260968s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3
12862024-08-28T02:10:50.039ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841") failed: Property not found. retry in 971.262183ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841
12872024-08-28T02:10:50.118ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_50ac3e36-ea5c-4742-b23f-aa0a78804739") failed: Property not found. retry in 675.206478ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_50ac3e36-ea5c-4742-b23f-aa0a78804739
12882024-08-28T02:10:50.263ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_7d4cdbf9-0586-4115-afd4-11bd8cc4917a") failed: Property not found. retry in 1.024477084s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_7d4cdbf9-0586-4115-afd4-11bd8cc4917a
12892024-08-28T02:10:50.293ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_f07808bb-11dd-4aef-b8f8-6a8eaddfe9dd") failed: Property not found. retry in 513.473679ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_f07808bb-11dd-4aef-b8f8-6a8eaddfe9dd
12902024-08-28T02:10:50.416ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859") failed: Property not found. retry in 1.200114244s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859
12912024-08-28T02:10:50.456ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06") failed: Property not found. retry in 834.064777ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06
12922024-08-28T02:10:50.472ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97") failed: Property not found. retry in 631.020007ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97
12932024-08-28T02:10:50.519ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30") failed: Property not found. retry in 1.414778833s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30
12942024-08-28T02:10:50.588ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a94447d4-8244-466f-894d-de67767ef978") failed: Property not found. retry in 970.657372ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a94447d4-8244-466f-894d-de67767ef978
12952024-08-28T02:10:50.677ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f") failed: Property not found. retry in 1.130255996s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f
12962024-08-28T02:10:50.710ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7a899aef-896e-41b9-8810-6c5024da5a0c") failed: Property not found. retry in 896.495192ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7a899aef-896e-41b9-8810-6c5024da5a0c
12972024-08-28T02:10:50.765ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf") failed: Property not found. retry in 1.211115017s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf
12982024-08-28T02:10:50.789ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_c359bc38-954c-4b35-9322-26531287be03") failed: Property not found. retry in 1.13850941s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_c359bc38-954c-4b35-9322-26531287be03
12992024-08-28T02:10:50.807ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_50ac3e36-ea5c-4742-b23f-aa0a78804739") failed: Property not found. retry in 642.159957ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_50ac3e36-ea5c-4742-b23f-aa0a78804739
13002024-08-28T02:10:50.818ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_f07808bb-11dd-4aef-b8f8-6a8eaddfe9dd") failed: Property not found. retry in 1.480842268s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_f07808bb-11dd-4aef-b8f8-6a8eaddfe9dd
13012024-08-28T02:10:50.828ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_87cde93b-53d3-4924-847b-c181020478b1") failed: Property not found. retry in 800.758042ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_87cde93b-53d3-4924-847b-c181020478b1
13022024-08-28T02:10:50.840ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4") failed: Property not found. retry in 532.179884ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4
13032024-08-28T02:10:50.905ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191") failed: Property not found. retry in 751.659568ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191
13042024-08-28T02:10:50.998ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded") failed: Property not found. retry in 765.664349ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded
13052024-08-28T02:10:51.026ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841") failed: Property not found. retry in 512.478258ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841
13062024-08-28T02:10:51.117ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97") failed: Property not found. retry in 704.803742ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97
13072024-08-28T02:10:51.132ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04") failed: Property not found. retry in 732.553946ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04
13082024-08-28T02:10:51.306ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_7d4cdbf9-0586-4115-afd4-11bd8cc4917a") failed: Property not found. retry in 848.057487ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_7d4cdbf9-0586-4115-afd4-11bd8cc4917a
13092024-08-28T02:10:51.319ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06") failed: Property not found. retry in 1.395905822s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06
13102024-08-28T02:10:51.336ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3") failed: Property not found. retry in 1.469638681s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3
13112024-08-28T02:10:51.388ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4") failed: Property not found. retry in 994.77843ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4
13122024-08-28T02:10:51.463ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_50ac3e36-ea5c-4742-b23f-aa0a78804739") failed: Property not found. retry in 1.360278225s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_50ac3e36-ea5c-4742-b23f-aa0a78804739
13132024-08-28T02:10:51.554ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841") failed: Property not found. retry in 532.86034ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841
13142024-08-28T02:10:51.571ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a94447d4-8244-466f-894d-de67767ef978") failed: Property not found. retry in 1.240221787s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a94447d4-8244-466f-894d-de67767ef978
13152024-08-28T02:10:51.622ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7a899aef-896e-41b9-8810-6c5024da5a0c") failed: Property not found. retry in 1.314548801s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7a899aef-896e-41b9-8810-6c5024da5a0c
13162024-08-28T02:10:51.637ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859") failed: Property not found. retry in 673.903824ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859
13172024-08-28T02:10:51.649ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_87cde93b-53d3-4924-847b-c181020478b1") failed: Property not found. retry in 724.242378ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_87cde93b-53d3-4924-847b-c181020478b1
13182024-08-28T02:10:51.671ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191") failed: Property not found. retry in 592.036909ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191
13192024-08-28T02:10:51.783ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded") failed: Property not found. retry in 588.669934ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded
13202024-08-28T02:10:51.822ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f") failed: Property not found. retry in 621.496965ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f
13212024-08-28T02:10:51.842ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97") failed: Property not found. retry in 1.084517983s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97
13222024-08-28T02:10:51.880ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04") failed: Property not found. retry in 992.80431ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04
13232024-08-28T02:10:51.944ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_c359bc38-954c-4b35-9322-26531287be03") failed: Property not found. retry in 1.273927677s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_c359bc38-954c-4b35-9322-26531287be03
13242024-08-28T02:10:51.962ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30") failed: Property not found. retry in 541.209485ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30
13252024-08-28T02:10:51.994ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf") failed: Property not found. retry in 702.278242ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf
13262024-08-28T02:10:52.103ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841") failed: Property not found. retry in 1.498454765s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841
13272024-08-28T02:10:52.171ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_7d4cdbf9-0586-4115-afd4-11bd8cc4917a") failed: Property not found. retry in 778.901564ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_7d4cdbf9-0586-4115-afd4-11bd8cc4917a
13282024-08-28T02:10:52.279ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191") failed: Property not found. retry in 1.02536165s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191
13292024-08-28T02:10:52.314ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_f07808bb-11dd-4aef-b8f8-6a8eaddfe9dd") failed: Property not found. retry in 1.263968765s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_f07808bb-11dd-4aef-b8f8-6a8eaddfe9dd
13302024-08-28T02:10:52.327ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859") failed: Property not found. retry in 543.665512ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859
13312024-08-28T02:10:52.386ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded") failed: Property not found. retry in 1.292460153s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded
13322024-08-28T02:10:52.405ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_87cde93b-53d3-4924-847b-c181020478b1") failed: Property not found. retry in 1.378566135s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_87cde93b-53d3-4924-847b-c181020478b1
13332024-08-28T02:10:52.423ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4") failed: Property not found. retry in 1.314064007s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4
13342024-08-28T02:10:52.458ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f") failed: Property not found. retry in 566.819631ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f
13352024-08-28T02:10:52.521ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30") failed: Property not found. retry in 964.707691ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30
13362024-08-28T02:10:52.710ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf") failed: Property not found. retry in 1.226707185s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf
13372024-08-28T02:10:52.730ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06") failed: Property not found. retry in 1.368994164s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06
13382024-08-28T02:10:52.817ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3") failed: Property not found. retry in 1.195183149s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3
13392024-08-28T02:10:52.828ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a94447d4-8244-466f-894d-de67767ef978") failed: Property not found. retry in 612.152338ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a94447d4-8244-466f-894d-de67767ef978
13402024-08-28T02:10:52.842ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_50ac3e36-ea5c-4742-b23f-aa0a78804739") failed: Property not found. retry in 1.009354482s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_50ac3e36-ea5c-4742-b23f-aa0a78804739
13412024-08-28T02:10:52.884ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859") failed: Property not found. retry in 950.88859ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859
13422024-08-28T02:10:52.896ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04") failed: Property not found. retry in 1.371154218s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04
13432024-08-28T02:10:52.937ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97") failed: Property not found. retry in 1.454158404s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97
13442024-08-28T02:10:52.948ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7a899aef-896e-41b9-8810-6c5024da5a0c") failed: Property not found. retry in 1.486308592s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7a899aef-896e-41b9-8810-6c5024da5a0c
13452024-08-28T02:10:52.959ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_7d4cdbf9-0586-4115-afd4-11bd8cc4917a") failed: Property not found. retry in 522.097059ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_7d4cdbf9-0586-4115-afd4-11bd8cc4917a
13462024-08-28T02:10:53.039ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f") failed: Property not found. retry in 1.118712921s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f
13472024-08-28T02:10:53.043ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 5.043346943s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers): operation timed out" file = oximeter/producer/src/lib.rs:424 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261
13482024-08-28T02:10:53.234ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_c359bc38-954c-4b35-9322-26531287be03") failed: Property not found. retry in 912.249364ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_c359bc38-954c-4b35-9322-26531287be03
13492024-08-28T02:10:53.505ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_7d4cdbf9-0586-4115-afd4-11bd8cc4917a") failed: Property not found. retry in 889.259666ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_7d4cdbf9-0586-4115-afd4-11bd8cc4917a
13502024-08-28T02:10:53.529ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30") failed: Property not found. retry in 595.844508ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30
13512024-08-28T02:10:53.709ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded") failed: Property not found. retry in 1.126527039s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded
13522024-08-28T02:10:53.862ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859") failed: Property not found. retry in 1.251984147s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859
13532024-08-28T02:10:53.921ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_50ac3e36-ea5c-4742-b23f-aa0a78804739") failed: Property not found. retry in 799.806813ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_50ac3e36-ea5c-4742-b23f-aa0a78804739
13542024-08-28T02:10:55.194ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_external_dns_6926ac43-4e2e-4378-9bce-42019ec72a30
13552024-08-28T02:10:55.194ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_crucible_15a449aa-64f1-460c-bd5b-f4eb65dd01a3
13562024-08-28T02:10:55.194ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_crucible_5b222c53-9b3a-4a9c-b345-0bfa38130f06
13572024-08-28T02:10:55.194ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_nexus_87cde93b-53d3-4924-847b-c181020478b1
13582024-08-28T02:10:55.194ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_nexus_c359bc38-954c-4b35-9322-26531287be03
13592024-08-28T02:10:55.194ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_crucible_b54f97d3-e521-4f5f-b20d-89bdd6114d3f
13602024-08-28T02:10:55.194ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_crucible_e475d3e7-6b31-42b3-a2d4-0f0fc3753ded
13612024-08-28T02:10:55.194ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_crucible_a94447d4-8244-466f-894d-de67767ef978
13622024-08-28T02:10:55.194ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_crucible_c1995310-e5ce-4e60-a5e1-46140ed4eeaf
13632024-08-28T02:10:55.194ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_oximeter_1d02a2ca-a68b-4b79-b6c3-45dfe594b841
13642024-08-28T02:10:55.194ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_crucible_72467e25-1520-4e5a-8e68-1eac43721859
13652024-08-28T02:10:55.194ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_crucible_pantry_9d29a24e-a625-4e1e-8a20-b16aadbb38f4
13662024-08-28T02:10:55.194ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_nexus_50ac3e36-ea5c-4742-b23f-aa0a78804739
13672024-08-28T02:10:55.194ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_crucible_fea45bee-76ac-4f8d-9161-d2363f2b6f04
13682024-08-28T02:10:55.194ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_crucible_pantry_a9beca1c-9fcd-4a3d-9cde-db0219285191
13692024-08-28T02:10:55.195ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_external_dns_7a899aef-896e-41b9-8810-6c5024da5a0c
13702024-08-28T02:10:55.195ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_crucible_c54505c9-f5cb-419b-8571-5173c8d0bb97
13712024-08-28T02:10:55.195ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_crucible_pantry_f07808bb-11dd-4aef-b8f8-6a8eaddfe9dd
13722024-08-28T02:10:55.195ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3245 zone = oxz_clickhouse_7d4cdbf9-0586-4115-afd4-11bd8cc4917a
13732024-08-28T02:10:55.196ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/3dd1ecc3-6cd1-48fe-8e10-622a58ff4257/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
13742024-08-28T02:10:55.197ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/fadb702a-2570-430e-a63c-e4ed78dd68d2/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
13752024-08-28T02:10:55.202ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 35805646 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:34415 req_id = a71df855-b72d-42a0-85c1-45b895ea9bd6 response_code = 204 uri = /omicron-zones
13762024-08-28T02:10:55.223ZINFOSledAgent (RSS): Finished setting up services file = sled-agent/src/rack_setup/service.rs:1286
13772024-08-28T02:10:55.223ZINFOSledAgent (RSS): Writing ledger to /pool/int/3dd1ecc3-6cd1-48fe-8e10-622a58ff4257/config/.rss-plan-completed.marker.tmp file = common/src/ledger.rs:199
13782024-08-28T02:10:55.223ZINFOSledAgent (RSS): Writing ledger to /pool/int/fadb702a-2570-430e-a63c-e4ed78dd68d2/config/.rss-plan-completed.marker.tmp file = common/src/ledger.rs:199
13792024-08-28T02:10:55.228ZINFOSledAgent (RSS): Handing off control to Nexus file = sled-agent/src/rack_setup/service.rs:687
13802024-08-28T02:10:55.235ZINFOSledAgent (RSS): Nexus address: [fd00:1122:3344:101::b]:12221 file = sled-agent/src/rack_setup/service.rs:700
13812024-08-28T02:10:55.299ZINFOSledAgent (RSS): rack_network_config: RackNetworkConfigV2 { bfd: [], bgp: [], infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [ PortConfigV2 { addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.2.174, width: 24, }, ), vlan_id: None, }, ], autoneg: false, bgp_peers: [], lldp: None, port: "qsfp0", routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), local_pref: None, nexthop: 10.151.2.100, vlan_id: None, }, ], switch: Switch0, uplink_port_fec: None, uplink_port_speed: Speed40G, }, ], rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, } file = sled-agent/src/rack_setup/service.rs:867
13822024-08-28T02:10:55.307ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/0c5c13d7-1d6f-4932-bd82-66253028d19d/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13832024-08-28T02:10:55.366ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/0c5c13d7-1d6f-4932-bd82-66253028d19d/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13842024-08-28T02:10:55.522ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/0c5c13d7-1d6f-4932-bd82-66253028d19d/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13852024-08-28T02:10:55.623ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/0c5c13d7-1d6f-4932-bd82-66253028d19d/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13862024-08-28T02:10:55.861ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/0c5c13d7-1d6f-4932-bd82-66253028d19d/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13872024-08-28T02:10:55.967ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/0c5c13d7-1d6f-4932-bd82-66253028d19d/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13882024-08-28T02:10:56.263ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/0c5c13d7-1d6f-4932-bd82-66253028d19d/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13892024-08-28T02:10:56.445ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/0c5c13d7-1d6f-4932-bd82-66253028d19d/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13902024-08-28T02:10:56.949ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/0c5c13d7-1d6f-4932-bd82-66253028d19d/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13912024-08-28T02:10:57.285ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/0c5c13d7-1d6f-4932-bd82-66253028d19d/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13922024-08-28T02:10:57.687ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/0c5c13d7-1d6f-4932-bd82-66253028d19d/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13932024-08-28T02:10:58.121ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 5.865744611s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers): error trying to connect: tcp connect error: Connection refused (os error 146)" file = oximeter/producer/src/lib.rs:424 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261
13942024-08-28T02:10:58.224ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/0c5c13d7-1d6f-4932-bd82-66253028d19d/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13952024-08-28T02:10:58.669ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/0c5c13d7-1d6f-4932-bd82-66253028d19d/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13962024-08-28T02:10:59.551ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/0c5c13d7-1d6f-4932-bd82-66253028d19d/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13972024-08-28T02:11:00.842ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/0c5c13d7-1d6f-4932-bd82-66253028d19d/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13982024-08-28T02:11:01.636ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/0c5c13d7-1d6f-4932-bd82-66253028d19d/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
13992024-08-28T02:11:02.489ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/0c5c13d7-1d6f-4932-bd82-66253028d19d/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
14002024-08-28T02:11:03.904ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/0c5c13d7-1d6f-4932-bd82-66253028d19d/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
14012024-08-28T02:11:04.020ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 19.18258057s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers): error trying to connect: tcp connect error: Connection refused (os error 146)" file = oximeter/producer/src/lib.rs:424 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261
14022024-08-28T02:11:05.761ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/0c5c13d7-1d6f-4932-bd82-66253028d19d/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
14032024-08-28T02:11:07.987ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/0c5c13d7-1d6f-4932-bd82-66253028d19d/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
14042024-08-28T02:11:11.898ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/0c5c13d7-1d6f-4932-bd82-66253028d19d/initialization-complete): error trying to connect: tcp connect error: Connection refused (os error 146) file = sled-agent/src/rack_setup/service.rs:929
14052024-08-28T02:11:18.272ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "54d6a7f8-6fdb-444c-995b-5d40ebd7667e", "content-length": "173", "date": "Wed, 28 Aug 2024 02:11:17 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"40b49400-5cd0-4d5c-a006-b241a1ca8261\"", request_id: "54d6a7f8-6fdb-444c-995b-5d40ebd7667e" } file = sled-agent/src/rack_setup/service.rs:929
14062024-08-28T02:11:19.464ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:39914
14072024-08-28T02:11:19.464ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 663 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:39914 req_id = aaae4446-aa77-4289-85f9-bd23835f456f response_code = 200 uri = /baseboard
14082024-08-28T02:11:22.573ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "9440929f-92ad-4fbb-8d8b-dcc0c788613c", "content-length": "173", "date": "Wed, 28 Aug 2024 02:11:22 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"40b49400-5cd0-4d5c-a006-b241a1ca8261\"", request_id: "9440929f-92ad-4fbb-8d8b-dcc0c788613c" } file = sled-agent/src/rack_setup/service.rs:929
14092024-08-28T02:11:29.401ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "b83cfa81-3c22-474b-8478-a532964c786b", "content-length": "173", "date": "Wed, 28 Aug 2024 02:11:29 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"40b49400-5cd0-4d5c-a006-b241a1ca8261\"", request_id: "b83cfa81-3c22-474b-8478-a532964c786b" } file = sled-agent/src/rack_setup/service.rs:929
14102024-08-28T02:11:30.930ZINFOSledAgent (NexusNotifierTask): Successfully put SledAgentInfo to nexus file = sled-agent/src/nexus.rs:544 new_generation = 1 old_generation = 1 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261
14112024-08-28T02:11:31.068ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:39519
14122024-08-28T02:11:31.069ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
14132024-08-28T02:11:31.151ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 83191 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:39519 req_id = 430f38ce-821a-4cc5-8811-575302172c90 response_code = 200 uri = /inventory
14142024-08-28T02:11:31.152ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
14152024-08-28T02:11:31.152ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/3dd1ecc3-6cd1-48fe-8e10-622a58ff4257/config/omicron-zones.json file = common/src/ledger.rs:177
14162024-08-28T02:11:31.153ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/fadb702a-2570-430e-a63c-e4ed78dd68d2/config/omicron-zones.json file = common/src/ledger.rs:177
14172024-08-28T02:11:31.154ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1790 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:39519 req_id = f478e228-e935-427a-9944-e49b247e60fd response_code = 200 uri = /omicron-zones
14182024-08-28T02:11:35.339ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:53217
14192024-08-28T02:11:35.341ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 16f83236-bc4a-4d30-8cf5-b8e748fe9792, kind: Service { id: 6926ac43-4e2e-4378-9bce-42019ec72a30 }, name: Name("external-dns-6926ac43-4e2e-4378-9bce-42019ec72a30"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 235, 144])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 67a6854e-59a4-4c1d-b17a-a4ec8e40b976, kind: Service { id: 7a899aef-896e-41b9-8810-6c5024da5a0c }, name: Name("external-dns-7a899aef-896e-41b9-8810-6c5024da5a0c"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 136, 51])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 138b549d-922d-4976-b2e2-89111b890ac2, kind: Service { id: c359bc38-954c-4b35-9322-26531287be03 }, name: Name("nexus-c359bc38-954c-4b35-9322-26531287be03"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 179, 102])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a81dd239-4fe9-4ce3-b19d-4876383286d2, kind: Service { id: 87cde93b-53d3-4924-847b-c181020478b1 }, name: Name("nexus-87cde93b-53d3-4924-847b-c181020478b1"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 249, 85])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cda1bc8f-c147-42a1-98e0-40955e79f4cf, kind: Service { id: 50ac3e36-ea5c-4742-b23f-aa0a78804739 }, name: Name("nexus-50ac3e36-ea5c-4742-b23f-aa0a78804739"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 180])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
14202024-08-28T02:11:35.341ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
14212024-08-28T02:11:35.341ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
14222024-08-28T02:11:35.341ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
14232024-08-28T02:11:35.341ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
14242024-08-28T02:11:35.341ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
14252024-08-28T02:11:35.341ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
14262024-08-28T02:11:35.341ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1839 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:53217 req_id = 2d77b145-f5ab-4b23-a5d1-6a80d05cb31b response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
14272024-08-28T02:11:35.343ZINFOSledAgent (RSS): Handoff to Nexus is complete file = sled-agent/src/rack_setup/service.rs:939
14282024-08-28T02:11:35.444ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:45537
14292024-08-28T02:11:35.444ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 16f83236-bc4a-4d30-8cf5-b8e748fe9792, kind: Service { id: 6926ac43-4e2e-4378-9bce-42019ec72a30 }, name: Name("external-dns-6926ac43-4e2e-4378-9bce-42019ec72a30"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 235, 144])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 67a6854e-59a4-4c1d-b17a-a4ec8e40b976, kind: Service { id: 7a899aef-896e-41b9-8810-6c5024da5a0c }, name: Name("external-dns-7a899aef-896e-41b9-8810-6c5024da5a0c"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 136, 51])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 138b549d-922d-4976-b2e2-89111b890ac2, kind: Service { id: c359bc38-954c-4b35-9322-26531287be03 }, name: Name("nexus-c359bc38-954c-4b35-9322-26531287be03"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 179, 102])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a81dd239-4fe9-4ce3-b19d-4876383286d2, kind: Service { id: 87cde93b-53d3-4924-847b-c181020478b1 }, name: Name("nexus-87cde93b-53d3-4924-847b-c181020478b1"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 249, 85])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cda1bc8f-c147-42a1-98e0-40955e79f4cf, kind: Service { id: 50ac3e36-ea5c-4742-b23f-aa0a78804739 }, name: Name("nexus-50ac3e36-ea5c-4742-b23f-aa0a78804739"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 180])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
14302024-08-28T02:11:35.444ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
14312024-08-28T02:11:35.444ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
14322024-08-28T02:11:35.444ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
14332024-08-28T02:11:35.444ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
14342024-08-28T02:11:35.444ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
14352024-08-28T02:11:35.444ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
14362024-08-28T02:11:35.444ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 246 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:45537 req_id = 7ab6a915-c306-4868-8825-2db9ca961799 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
14372024-08-28T02:11:35.562ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:49915
14382024-08-28T02:11:35.563ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 16f83236-bc4a-4d30-8cf5-b8e748fe9792, kind: Service { id: 6926ac43-4e2e-4378-9bce-42019ec72a30 }, name: Name("external-dns-6926ac43-4e2e-4378-9bce-42019ec72a30"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 235, 144])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 67a6854e-59a4-4c1d-b17a-a4ec8e40b976, kind: Service { id: 7a899aef-896e-41b9-8810-6c5024da5a0c }, name: Name("external-dns-7a899aef-896e-41b9-8810-6c5024da5a0c"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 136, 51])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 138b549d-922d-4976-b2e2-89111b890ac2, kind: Service { id: c359bc38-954c-4b35-9322-26531287be03 }, name: Name("nexus-c359bc38-954c-4b35-9322-26531287be03"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 179, 102])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a81dd239-4fe9-4ce3-b19d-4876383286d2, kind: Service { id: 87cde93b-53d3-4924-847b-c181020478b1 }, name: Name("nexus-87cde93b-53d3-4924-847b-c181020478b1"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 249, 85])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cda1bc8f-c147-42a1-98e0-40955e79f4cf, kind: Service { id: 50ac3e36-ea5c-4742-b23f-aa0a78804739 }, name: Name("nexus-50ac3e36-ea5c-4742-b23f-aa0a78804739"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 180])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
14392024-08-28T02:11:35.563ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
14402024-08-28T02:11:35.563ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
14412024-08-28T02:11:35.563ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
14422024-08-28T02:11:35.563ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
14432024-08-28T02:11:35.563ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
14442024-08-28T02:11:35.563ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
14452024-08-28T02:11:35.563ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 260 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:49915 req_id = 4b075cbc-9a36-45f6-b283-ad3ed593e7b3 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
14462024-08-28T02:11:35.630ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:55889
14472024-08-28T02:11:35.630ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
14482024-08-28T02:11:35.715ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 85074 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:55889 req_id = 2108c4b7-764f-4e9a-a85f-ea8c9251b9da response_code = 200 uri = /inventory
14492024-08-28T02:11:35.715ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
14502024-08-28T02:11:35.715ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/3dd1ecc3-6cd1-48fe-8e10-622a58ff4257/config/omicron-zones.json file = common/src/ledger.rs:177
14512024-08-28T02:11:35.716ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/fadb702a-2570-430e-a63c-e4ed78dd68d2/config/omicron-zones.json file = common/src/ledger.rs:177
14522024-08-28T02:11:35.716ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 943 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:55889 req_id = 42023eb4-146e-4e1f-a5e1-d78542ec3564 response_code = 200 uri = /omicron-zones
14532024-08-28T02:11:35.740ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:55451
14542024-08-28T02:11:35.740ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 16f83236-bc4a-4d30-8cf5-b8e748fe9792, kind: Service { id: 6926ac43-4e2e-4378-9bce-42019ec72a30 }, name: Name("external-dns-6926ac43-4e2e-4378-9bce-42019ec72a30"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 235, 144])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 67a6854e-59a4-4c1d-b17a-a4ec8e40b976, kind: Service { id: 7a899aef-896e-41b9-8810-6c5024da5a0c }, name: Name("external-dns-7a899aef-896e-41b9-8810-6c5024da5a0c"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 136, 51])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 138b549d-922d-4976-b2e2-89111b890ac2, kind: Service { id: c359bc38-954c-4b35-9322-26531287be03 }, name: Name("nexus-c359bc38-954c-4b35-9322-26531287be03"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 179, 102])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a81dd239-4fe9-4ce3-b19d-4876383286d2, kind: Service { id: 87cde93b-53d3-4924-847b-c181020478b1 }, name: Name("nexus-87cde93b-53d3-4924-847b-c181020478b1"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 249, 85])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cda1bc8f-c147-42a1-98e0-40955e79f4cf, kind: Service { id: 50ac3e36-ea5c-4742-b23f-aa0a78804739 }, name: Name("nexus-50ac3e36-ea5c-4742-b23f-aa0a78804739"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 180])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
14552024-08-28T02:11:35.740ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
14562024-08-28T02:11:35.740ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
14572024-08-28T02:11:35.740ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
14582024-08-28T02:11:35.740ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
14592024-08-28T02:11:35.740ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
14602024-08-28T02:11:35.740ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
14612024-08-28T02:11:35.740ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 252 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:55451 req_id = b25a6993-5f49-4bfe-af16-69f40302a4d6 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
14622024-08-28T02:11:41.547ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:64715
14632024-08-28T02:11:41.547ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 265 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:64715 req_id = ea6bf167-0b6c-4717-84f4-2f8acec6f029 response_code = 200 uri = /vpc-routes
14642024-08-28T02:11:41.758ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1538 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:64715 req_id = fa42f323-27c3-42cd-afb0-7a46315ee283 response_code = 204 uri = /vpc-routes
14652024-08-28T02:11:46.510ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:58827
14662024-08-28T02:11:46.510ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 84 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:58827 req_id = 6ef19972-8236-4e63-ae10-6b7437f89866 response_code = 200 uri = /vpc-routes
14672024-08-28T02:11:46.523ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 826 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:58827 req_id = a8da17fe-2299-487a-8916-1d92a2d1d320 response_code = 204 uri = /vpc-routes
14682024-08-28T02:11:46.526ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:56672
14692024-08-28T02:11:46.526ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 31 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:56672 req_id = 5298d350-8389-4eb5-a85c-822629c3a7fe response_code = 200 uri = /vpc-routes
14702024-08-28T02:11:46.538ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 103 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:56672 req_id = c5082708-0d6e-4211-94f3-991bed515736 response_code = 204 uri = /vpc-routes
14712024-08-28T02:11:46.637ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:34398
14722024-08-28T02:11:46.637ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 165 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:34398 req_id = 712cc94a-3148-4a59-ba63-94cb2b5d1b62 response_code = 200 uri = /v2p
14732024-08-28T02:11:46.660ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:38087
14742024-08-28T02:11:46.660ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 119 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:38087 req_id = 8122032f-1c1b-435d-bdce-47cf1ce61458 response_code = 200 uri = /v2p
14752024-08-28T02:11:46.719ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:62772
14762024-08-28T02:11:46.719ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
14772024-08-28T02:11:46.782ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
14782024-08-28T02:11:46.840ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 121342 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:62772 req_id = 679f5916-6d4c-4591-8666-f50697612362 response_code = 204 uri = /switch-ports
14792024-08-28T02:11:46.841ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:45590
14802024-08-28T02:11:46.841ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
14812024-08-28T02:11:46.847ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 5993 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:62772 req_id = 90d75f37-828a-4ff8-a764-e4983cf4641a response_code = 200 uri = /network-bootstore-config
14822024-08-28T02:11:46.880ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
14832024-08-28T02:11:46.886ZINFOSledAgent (bootstore): Attempting to update network config with generation: 2, current_generation: 1 file = bootstore/src/schemes/v0/peer.rs:564 peer_id = pc-sock-i86pc
14842024-08-28T02:11:46.886ZINFOSledAgent (bootstore): Writing ledger to /pool/int/3dd1ecc3-6cd1-48fe-8e10-622a58ff4257/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
14852024-08-28T02:11:46.888ZINFOSledAgent (bootstore): Writing ledger to /pool/int/fadb702a-2570-430e-a63c-e4ed78dd68d2/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
14862024-08-28T02:11:46.888ZINFOSledAgent (bootstore): Broadcasting network config with generation 2 file = bootstore/src/schemes/v0/peer.rs:648 peer_id = pc-sock-i86pc
14872024-08-28T02:11:46.888ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 4584 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:62772 req_id = 6d80043f-467e-4abd-86a8-3976166be000 response_code = 204 uri = /network-bootstore-config
14882024-08-28T02:11:46.936ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 95453 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:45590 req_id = 323c237d-a775-4bcf-8b6d-6b19c082795b response_code = 204 uri = /switch-ports
14892024-08-28T02:11:46.937ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 150 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:45590 req_id = 8078107f-5238-4644-af16-c800fa2ad98e response_code = 200 uri = /network-bootstore-config
14902024-08-28T02:11:47.483ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:47971
14912024-08-28T02:11:47.483ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 97 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:47971 req_id = 53d70bcd-a175-412a-a50c-e036461f0f76 response_code = 200 uri = /vpc-routes
14922024-08-28T02:11:47.561ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:38303
14932024-08-28T02:11:47.561ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 157 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:38303 req_id = 443de1f4-8d84-4b84-b277-5c3442a95c40 response_code = 200 uri = /v2p
14942024-08-28T02:11:47.565ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 117 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:47971 req_id = 9cc63a4c-d28d-4fbb-ad67-e149272e2886 response_code = 204 uri = /vpc-routes
14952024-08-28T02:11:47.693ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:32926
14962024-08-28T02:11:47.693ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
14972024-08-28T02:11:47.730ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
14982024-08-28T02:11:47.781ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 87707 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:32926 req_id = da902ffc-bc6e-4635-a7af-51c750e65ab9 response_code = 204 uri = /switch-ports
14992024-08-28T02:11:47.781ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 121 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:32926 req_id = 20cad4df-3bc3-483a-aa81-0b9e6257ea85 response_code = 200 uri = /network-bootstore-config
15002024-08-28T02:11:49.498ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:50328
15012024-08-28T02:11:49.498ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 93 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:50328 req_id = 2ed68bed-cd2b-4a38-9c98-fd0bb6a16c20 response_code = 200 uri = /baseboard
15022024-08-28T02:11:53.349ZINFOSledAgent (dropshot): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:49250 remote_addr = [fd00:1122:3344:101::d]:45985 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261
15032024-08-28T02:11:53.371ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 21434 local_addr = [fd00:1122:3344:101::1]:49250 method = GET remote_addr = [fd00:1122:3344:101::d]:45985 req_id = bd5e0715-8c8f-4928-914f-0978664fccd9 response_code = 200 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261 uri = /40b49400-5cd0-4d5c-a006-b241a1ca8261
15042024-08-28T02:12:16.572ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:40394
15052024-08-28T02:12:16.574ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 89 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:40394 req_id = cadbbdbd-ab24-4b03-993a-295bcac887a7 response_code = 200 uri = /vpc-routes
15062024-08-28T02:12:16.663ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:35558
15072024-08-28T02:12:16.664ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 62 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:35558 req_id = b0e83564-a5c4-4d7d-82ac-122a64238888 response_code = 200 uri = /vpc-routes
15082024-08-28T02:12:16.695ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 116 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:40394 req_id = 02af84a9-9b1b-402d-845d-8a2384109b39 response_code = 204 uri = /vpc-routes
15092024-08-28T02:12:16.736ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:47302
15102024-08-28T02:12:16.737ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1171 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:47302 req_id = 9f872749-692a-4edd-9385-cd333f7f7deb response_code = 200 uri = /v2p
15112024-08-28T02:12:16.745ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 75 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:35558 req_id = 1e4eff03-ebe5-4655-bdb5-fea2630be543 response_code = 204 uri = /vpc-routes
15122024-08-28T02:12:16.799ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:41831
15132024-08-28T02:12:16.799ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 90 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:41831 req_id = 6814af39-dde3-441f-b1c9-3e0b48ea1c14 response_code = 200 uri = /v2p
15142024-08-28T02:12:16.968ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:60872
15152024-08-28T02:12:16.968ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
15162024-08-28T02:12:17.007ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
15172024-08-28T02:12:17.065ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 97825 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:60872 req_id = b78bc3d1-32c6-4bd9-8f92-4376bb8479e2 response_code = 204 uri = /switch-ports
15182024-08-28T02:12:17.066ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 111 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:60872 req_id = 2a051c61-ab74-41c1-a5e1-8226038395cf response_code = 200 uri = /network-bootstore-config
15192024-08-28T02:12:17.156ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:38817
15202024-08-28T02:12:17.156ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
15212024-08-28T02:12:17.204ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
15222024-08-28T02:12:17.281ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 125456 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:38817 req_id = 24af4e07-6ed9-4b93-8283-a3e09f79d602 response_code = 204 uri = /switch-ports
15232024-08-28T02:12:17.282ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 119 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:38817 req_id = 525293bb-8cff-4085-b4dc-bb69f0f4411f response_code = 200 uri = /network-bootstore-config
15242024-08-28T02:12:17.547ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:55320
15252024-08-28T02:12:17.547ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 93 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:55320 req_id = fd31e218-8f90-4470-8647-22de4ce3a20f response_code = 200 uri = /vpc-routes
15262024-08-28T02:12:17.560ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 139 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:55320 req_id = 19056e4e-5778-426d-9803-409d9b841dc1 response_code = 204 uri = /vpc-routes
15272024-08-28T02:12:17.607ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:50087
15282024-08-28T02:12:17.608ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
15292024-08-28T02:12:17.644ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
15302024-08-28T02:12:17.700ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 92770 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:50087 req_id = 591c4013-d317-4046-b65c-9b433caeda24 response_code = 204 uri = /switch-ports
15312024-08-28T02:12:17.700ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:37960
15322024-08-28T02:12:17.701ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 743 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:37960 req_id = e0e5ff3f-c673-4483-9b6f-ece9891e5525 response_code = 200 uri = /v2p
15332024-08-28T02:12:17.701ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 92 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:50087 req_id = b0f67c97-c248-4777-9194-af889c46c403 response_code = 200 uri = /network-bootstore-config
15342024-08-28T02:12:19.538ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:35286
15352024-08-28T02:12:19.539ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 81 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:35286 req_id = cd4e4902-7738-4478-ab70-6e5a41db8198 response_code = 200 uri = /baseboard
15362024-08-28T02:12:23.358ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 8744 local_addr = [fd00:1122:3344:101::1]:49250 method = GET remote_addr = [fd00:1122:3344:101::d]:45985 req_id = 54364022-b711-4e6d-957b-74769c48d37a response_code = 200 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261 uri = /40b49400-5cd0-4d5c-a006-b241a1ca8261
15372024-08-28T02:12:46.475ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:52842
15382024-08-28T02:12:46.475ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 117 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:52842 req_id = 26d0c6a8-cd9b-4cf7-b261-e6008b3ab60e response_code = 200 uri = /vpc-routes
15392024-08-28T02:12:46.496ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:55686
15402024-08-28T02:12:46.496ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 89 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:55686 req_id = 3d828933-fa16-4eb3-a115-72709efdc29d response_code = 200 uri = /vpc-routes
15412024-08-28T02:12:46.499ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1125 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:52842 req_id = 55b4569b-4c2d-4c15-aaa1-f443cdc8d5d6 response_code = 204 uri = /vpc-routes
15422024-08-28T02:12:46.516ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 116 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:55686 req_id = dc9ef4ca-03b9-4694-8529-4ec5086e42b0 response_code = 204 uri = /vpc-routes
15432024-08-28T02:12:46.681ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:42137
15442024-08-28T02:12:46.681ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
15452024-08-28T02:12:46.721ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
15462024-08-28T02:12:46.775ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 94061 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:42137 req_id = 1a52fbd4-dd6d-4e88-8184-26f488f8bcb9 response_code = 204 uri = /switch-ports
15472024-08-28T02:12:46.776ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:36676
15482024-08-28T02:12:46.776ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:40419
15492024-08-28T02:12:46.776ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:33641
15502024-08-28T02:12:46.776ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
15512024-08-28T02:12:46.780ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 4199 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:42137 req_id = 030764fe-f8f5-4605-b4a1-d751d1e3b99d response_code = 200 uri = /network-bootstore-config
15522024-08-28T02:12:46.780ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 4333 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:36676 req_id = 9b5d1607-8618-4a06-8238-6afcd7d5e8f8 response_code = 200 uri = /v2p
15532024-08-28T02:12:46.780ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 4306 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:33641 req_id = 45b6e151-d280-4fc2-b948-1ebed149eb35 response_code = 200 uri = /v2p
15542024-08-28T02:12:46.811ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
15552024-08-28T02:12:46.863ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 87263 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:40419 req_id = 77243dfc-6f46-467c-8212-b575cf6cdf25 response_code = 204 uri = /switch-ports
15562024-08-28T02:12:46.863ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 141 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:40419 req_id = b10df438-89e5-4558-b2d3-6deaeb9d21c8 response_code = 200 uri = /network-bootstore-config
15572024-08-28T02:12:47.274ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:40060
15582024-08-28T02:12:47.275ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 79 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:40060 req_id = 1bae190b-9895-41c0-b0a0-afae342ee495 response_code = 200 uri = /vpc-routes
15592024-08-28T02:12:47.295ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 188 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:40060 req_id = d24c9c30-a757-4229-877f-61fc25c58a5e response_code = 204 uri = /vpc-routes
15602024-08-28T02:12:47.466ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:60025
15612024-08-28T02:12:47.467ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 125 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:60025 req_id = 3580bbec-ea69-4533-b663-78d13821799d response_code = 200 uri = /v2p
15622024-08-28T02:12:47.526ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:40440
15632024-08-28T02:12:47.526ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
15642024-08-28T02:12:47.565ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
15652024-08-28T02:12:47.621ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 94781 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:40440 req_id = c5fa2128-35ec-4129-97b7-a5a3ff5564cb response_code = 204 uri = /switch-ports
15662024-08-28T02:12:47.621ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 98 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:40440 req_id = fd221fa1-7626-42b7-95a7-fff88de63773 response_code = 200 uri = /network-bootstore-config
15672024-08-28T02:12:49.574ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:56889
15682024-08-28T02:12:49.575ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 62 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:56889 req_id = 52791716-f124-49f7-9b18-0204983a068e response_code = 200 uri = /baseboard
15692024-08-28T02:12:53.358ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 9493 local_addr = [fd00:1122:3344:101::1]:49250 method = GET remote_addr = [fd00:1122:3344:101::d]:45985 req_id = c5a8e5d0-8bd2-492f-9db2-0aaf89f3e84a response_code = 200 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261 uri = /40b49400-5cd0-4d5c-a006-b241a1ca8261
15702024-08-28T02:13:16.600ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:45505
15712024-08-28T02:13:16.600ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 82 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:45505 req_id = 63a5c42a-dcbc-400b-a210-2b3228fc6bab response_code = 200 uri = /vpc-routes
15722024-08-28T02:13:16.632ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:37663
15732024-08-28T02:13:16.632ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 152 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:37663 req_id = 135b9c82-d40e-4f91-bd10-fba39cb8debe response_code = 200 uri = /v2p
15742024-08-28T02:13:16.638ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:53864
15752024-08-28T02:13:16.638ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 40 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:53864 req_id = 9b595830-8cf9-41ef-86e5-07d72d09fab0 response_code = 200 uri = /vpc-routes
15762024-08-28T02:13:16.648ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 107 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:53864 req_id = b99fc3ba-7d02-4e66-b1c1-699c20921923 response_code = 204 uri = /vpc-routes
15772024-08-28T02:13:16.709ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:33984
15782024-08-28T02:13:16.710ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 995 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:33984 req_id = c55bd6a8-1c93-4e35-afc9-5405ee53f439 response_code = 200 uri = /v2p
15792024-08-28T02:13:16.718ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 70 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:45505 req_id = a283baff-ed77-4fda-b6e2-565e04b949a7 response_code = 204 uri = /vpc-routes
15802024-08-28T02:13:16.816ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:45581
15812024-08-28T02:13:16.816ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
15822024-08-28T02:13:16.853ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
15832024-08-28T02:13:16.907ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 90857 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:45581 req_id = e9102615-bb65-453e-9810-0bcf1d73c76f response_code = 204 uri = /switch-ports
15842024-08-28T02:13:16.907ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:63464
15852024-08-28T02:13:16.907ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
15862024-08-28T02:13:16.911ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3510 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:45581 req_id = c92a4c6e-c73c-4d2f-b995-cd8a6fbb3a34 response_code = 200 uri = /network-bootstore-config
15872024-08-28T02:13:16.940ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
15882024-08-28T02:13:16.991ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 84038 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:63464 req_id = 5b21302a-e1ad-4a51-8060-f75116b523b6 response_code = 204 uri = /switch-ports
15892024-08-28T02:13:16.992ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 141 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:63464 req_id = cb609b94-9fba-4e5c-937d-0fa80ad5e092 response_code = 200 uri = /network-bootstore-config
15902024-08-28T02:13:17.474ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:38788
15912024-08-28T02:13:17.474ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 90 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:38788 req_id = 19d527c8-e558-4d9f-9f70-5f890cfb32db response_code = 200 uri = /vpc-routes
15922024-08-28T02:13:17.501ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:41286
15932024-08-28T02:13:17.501ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 151 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:41286 req_id = 75f97077-eff0-4a16-92bf-895d6a4971d5 response_code = 200 uri = /v2p
15942024-08-28T02:13:17.508ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 99 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:38788 req_id = e50d6c97-7474-4edd-a72f-c91f398d7305 response_code = 204 uri = /vpc-routes
15952024-08-28T02:13:17.676ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:52910
15962024-08-28T02:13:17.676ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
15972024-08-28T02:13:17.713ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
15982024-08-28T02:13:17.767ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 91143 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:52910 req_id = a4899942-4952-4fe5-bc92-230017015fd2 response_code = 204 uri = /switch-ports
15992024-08-28T02:13:17.768ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 766 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:52910 req_id = ed399622-0386-46fc-b812-f0817ad4ef13 response_code = 200 uri = /network-bootstore-config
16002024-08-28T02:13:19.612ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:63123
16012024-08-28T02:13:19.613ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 549 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:63123 req_id = 6e69f55d-b04e-46e7-b240-e5a9ff077155 response_code = 200 uri = /baseboard
16022024-08-28T02:13:23.356ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 7046 local_addr = [fd00:1122:3344:101::1]:49250 method = GET remote_addr = [fd00:1122:3344:101::d]:45985 req_id = 0f5f174b-13d0-41e8-a2d3-94f9b4792672 response_code = 200 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261 uri = /40b49400-5cd0-4d5c-a006-b241a1ca8261
16032024-08-28T02:13:46.404ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:33160
16042024-08-28T02:13:46.404ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 107 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:33160 req_id = 9abcc5e1-e542-4b71-80f2-8e7b41d0a174 response_code = 200 uri = /vpc-routes
16052024-08-28T02:13:46.411ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:43654
16062024-08-28T02:13:46.411ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 39 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:43654 req_id = 7ba58c63-fa63-48c0-9c69-849d1c0d7cf9 response_code = 200 uri = /vpc-routes
16072024-08-28T02:13:46.427ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 137 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:43654 req_id = 5b0a9238-11d0-4bdb-9c30-bb480a0638e9 response_code = 204 uri = /vpc-routes
16082024-08-28T02:13:46.477ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:33494
16092024-08-28T02:13:46.477ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 151 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:33494 req_id = 27a58bc2-3d8b-4f9f-8241-51a713e64842 response_code = 200 uri = /v2p
16102024-08-28T02:13:46.487ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 87 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:33160 req_id = 070750a4-c74b-435b-93af-cd0994c72a5a response_code = 204 uri = /vpc-routes
16112024-08-28T02:13:46.570ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:40641
16122024-08-28T02:13:46.571ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 122 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:40641 req_id = de71110c-8d9e-4dc8-a6cb-3087542fe2b8 response_code = 200 uri = /v2p
16132024-08-28T02:13:46.593ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:50119
16142024-08-28T02:13:46.594ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
16152024-08-28T02:13:46.631ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
16162024-08-28T02:13:46.689ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 95976 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:50119 req_id = 8afe6a0f-7508-4751-9407-5025cc1f65bb response_code = 204 uri = /switch-ports
16172024-08-28T02:13:46.690ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:64164
16182024-08-28T02:13:46.690ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
16192024-08-28T02:13:46.693ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 3302 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:50119 req_id = cefa3c17-1ba9-4651-b8c5-e8905e6ad469 response_code = 200 uri = /network-bootstore-config
16202024-08-28T02:13:46.727ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
16212024-08-28T02:13:46.779ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 89759 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:64164 req_id = 86d15167-371c-4efd-84f6-0bb91f3ff7ad response_code = 204 uri = /switch-ports
16222024-08-28T02:13:46.781ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 824 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:64164 req_id = 4715ae95-6fb2-4a4c-a0f1-c7b4f7366b2b response_code = 200 uri = /network-bootstore-config
16232024-08-28T02:13:47.396ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:43741
16242024-08-28T02:13:47.397ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 829 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:43741 req_id = b5021029-5729-4fed-aff0-e7557d28f4d2 response_code = 200 uri = /v2p
16252024-08-28T02:13:47.401ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:52379
16262024-08-28T02:13:47.401ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 45 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:52379 req_id = 16ef9a65-1b8f-4cc1-9267-f5c76e740b2b response_code = 200 uri = /vpc-routes
16272024-08-28T02:13:47.471ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:36966
16282024-08-28T02:13:47.471ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
16292024-08-28T02:13:47.507ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
16302024-08-28T02:13:47.561ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 90625 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:36966 req_id = 7d136789-425c-4e6b-b8ba-e2d974ea19e8 response_code = 204 uri = /switch-ports
16312024-08-28T02:13:47.562ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 101 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:52379 req_id = 9d45ba82-ed5b-4956-934f-c2c90cb456ce response_code = 204 uri = /vpc-routes
16322024-08-28T02:13:47.562ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 712 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:36966 req_id = 1f44a86c-4235-46c4-a0bc-28271795a097 response_code = 200 uri = /network-bootstore-config
16332024-08-28T02:13:49.645ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:50291
16342024-08-28T02:13:49.646ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 98 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:50291 req_id = abb32efe-cc43-4d5b-b01b-8ac76a892a64 response_code = 200 uri = /baseboard
16352024-08-28T02:13:53.370ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 12368 local_addr = [fd00:1122:3344:101::1]:49250 method = GET remote_addr = [fd00:1122:3344:101::d]:45985 req_id = 5ba59f4f-baf4-4a67-81dd-3ad6779d5de8 response_code = 200 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261 uri = /40b49400-5cd0-4d5c-a006-b241a1ca8261
16362024-08-28T02:14:04.620ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
16372024-08-28T02:14:16.557ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:38337
16382024-08-28T02:14:16.557ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 133 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:38337 req_id = 9f89347f-68a8-429d-9df0-42ffe86b4425 response_code = 200 uri = /vpc-routes
16392024-08-28T02:14:16.692ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 163 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:38337 req_id = 3a47b066-731e-44a9-a3ea-7e55e4606111 response_code = 204 uri = /vpc-routes
16402024-08-28T02:14:16.766ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:53307
16412024-08-28T02:14:16.766ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 125 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:53307 req_id = 68d942cb-5185-418e-9052-7aad6aba9005 response_code = 200 uri = /v2p
16422024-08-28T02:14:16.788ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:64939
16432024-08-28T02:14:16.788ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 37 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:64939 req_id = edbb00fe-fecd-4552-a62b-22af121f368d response_code = 200 uri = /vpc-routes
16442024-08-28T02:14:16.801ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 91 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:64939 req_id = 6fb8c37c-fd51-4516-a06b-2bcef2344b92 response_code = 204 uri = /vpc-routes
16452024-08-28T02:14:16.814ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:49284
16462024-08-28T02:14:16.814ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
16472024-08-28T02:14:16.852ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
16482024-08-28T02:14:16.910ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 96029 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:49284 req_id = 13d4d6ce-5e8e-48a3-828d-ff6211fbca8a response_code = 204 uri = /switch-ports
16492024-08-28T02:14:16.911ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:61052
16502024-08-28T02:14:16.911ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:38349
16512024-08-28T02:14:16.912ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
16522024-08-28T02:14:16.915ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 4466 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:49284 req_id = ddba1bf2-5e0a-4ab2-b39a-70dd4bce3c25 response_code = 200 uri = /network-bootstore-config
16532024-08-28T02:14:16.919ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 4338 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:61052 req_id = adcd5bfe-c84e-4cd0-84ce-52ab0109e2a5 response_code = 200 uri = /v2p
16542024-08-28T02:14:16.947ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
16552024-08-28T02:14:16.998ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 86728 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:38349 req_id = 0d4b5a94-2767-44e2-8c23-6c2c4f775b0c response_code = 204 uri = /switch-ports
16562024-08-28T02:14:16.998ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 158 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:38349 req_id = a75baada-4f26-497f-89ae-5e073abcf531 response_code = 200 uri = /network-bootstore-config
16572024-08-28T02:14:17.361ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:48258
16582024-08-28T02:14:17.361ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 88 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:48258 req_id = e3864fa9-275f-4b95-a7b4-2f09180bfa5b response_code = 200 uri = /vpc-routes
16592024-08-28T02:14:17.375ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 124 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:48258 req_id = bc285620-d028-4278-b91e-6391d78e91d7 response_code = 204 uri = /vpc-routes
16602024-08-28T02:14:17.608ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:40498
16612024-08-28T02:14:17.608ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 154 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:40498 req_id = b2df2582-2f2e-4494-8b8f-7a10777985ae response_code = 200 uri = /v2p
16622024-08-28T02:14:17.713ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:43371
16632024-08-28T02:14:17.714ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
16642024-08-28T02:14:17.751ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
16652024-08-28T02:14:17.806ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 92348 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:43371 req_id = c5ebd987-ea1f-4f41-9aa5-1be2e63314f4 response_code = 204 uri = /switch-ports
16662024-08-28T02:14:17.806ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 108 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:43371 req_id = b2bf7096-19ff-4f84-bde5-6e6a73644c45 response_code = 200 uri = /network-bootstore-config
16672024-08-28T02:14:19.678ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:41914
16682024-08-28T02:14:19.678ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 99 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:41914 req_id = a6f5fb40-f2d7-499a-9bb8-99af74da8417 response_code = 200 uri = /baseboard
16692024-08-28T02:14:23.455ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 5989 local_addr = [fd00:1122:3344:101::1]:49250 method = GET remote_addr = [fd00:1122:3344:101::d]:45985 req_id = 3c0f030d-d9d1-4d1b-ba50-21588105fae0 response_code = 200 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261 uri = /40b49400-5cd0-4d5c-a006-b241a1ca8261
16702024-08-28T02:14:46.540ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:51151
16712024-08-28T02:14:46.545ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:38112
16722024-08-28T02:14:46.545ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 4795 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:51151 req_id = 3057ff87-2092-43a2-b2f7-c6fb9d267fbe response_code = 200 uri = /vpc-routes
16732024-08-28T02:14:46.545ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 245 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:38112 req_id = 1bff09f0-1a78-4829-8d20-cd0b46246542 response_code = 200 uri = /vpc-routes
16742024-08-28T02:14:46.555ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 154 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:38112 req_id = 3c88c20f-7c6c-426b-9ffe-8937944f26f9 response_code = 204 uri = /vpc-routes
16752024-08-28T02:14:46.602ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:52186
16762024-08-28T02:14:46.602ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
16772024-08-28T02:14:46.642ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
16782024-08-28T02:14:46.705ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 102696 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:52186 req_id = c94a8e53-1ee8-4237-8493-767d9f7dae9a response_code = 204 uri = /switch-ports
16792024-08-28T02:14:46.705ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:46895
16802024-08-28T02:14:46.705ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:39963
16812024-08-28T02:14:46.705ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:41732
16822024-08-28T02:14:46.706ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 364 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:46895 req_id = ce70bfda-cbe5-401c-9120-fc69dfe4510e response_code = 200 uri = /v2p
16832024-08-28T02:14:46.706ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 361 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:51151 req_id = 7e708494-6d66-467e-b356-b3dcd77acdf8 response_code = 204 uri = /vpc-routes
16842024-08-28T02:14:46.706ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
16852024-08-28T02:14:46.706ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 298 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:39963 req_id = e4b3e0d8-2818-4454-84cd-35aad4551c62 response_code = 200 uri = /v2p
16862024-08-28T02:14:46.711ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 5735 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:52186 req_id = b4870d5b-3758-4379-bfb3-604c7240b079 response_code = 200 uri = /network-bootstore-config
16872024-08-28T02:14:46.746ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
16882024-08-28T02:14:46.800ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 94435 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:41732 req_id = 5c12f7dc-0b01-4712-951f-280208a1b5c6 response_code = 204 uri = /switch-ports
16892024-08-28T02:14:46.801ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 166 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:41732 req_id = a90b7a49-da17-4266-9c57-2b8457c3ef0a response_code = 200 uri = /network-bootstore-config
16902024-08-28T02:14:47.341ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:50928
16912024-08-28T02:14:47.342ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 888 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:50928 req_id = b9100ec1-6a91-4d48-a4a0-ba33df69afe1 response_code = 200 uri = /vpc-routes
16922024-08-28T02:14:47.404ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:33737
16932024-08-28T02:14:47.404ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 159 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:33737 req_id = b718971f-4084-4762-847d-ebe4cd0384cb response_code = 200 uri = /v2p
16942024-08-28T02:14:47.414ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 108 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:50928 req_id = c28a3bbc-27fd-4773-88a7-9a69f05fa51d response_code = 204 uri = /vpc-routes
16952024-08-28T02:14:47.472ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:59792
16962024-08-28T02:14:47.472ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
16972024-08-28T02:14:47.515ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
16982024-08-28T02:14:47.581ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 108536 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:59792 req_id = d5cfe532-b9c1-4625-9300-2fbd50c08cf8 response_code = 204 uri = /switch-ports
16992024-08-28T02:14:47.582ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1184 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:59792 req_id = 9a0458fa-8f51-47b5-8879-1b373be045d6 response_code = 200 uri = /network-bootstore-config
17002024-08-28T02:14:49.712ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:33229
17012024-08-28T02:14:49.712ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 95 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:33229 req_id = 746f3538-0cc4-4030-9302-60f933586b0e response_code = 200 uri = /baseboard
17022024-08-28T02:14:53.454ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 5338 local_addr = [fd00:1122:3344:101::1]:49250 method = GET remote_addr = [fd00:1122:3344:101::d]:45985 req_id = b1a44a0d-2f06-4701-9db0-1fe11dc92fe9 response_code = 200 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261 uri = /40b49400-5cd0-4d5c-a006-b241a1ca8261
17032024-08-28T02:15:16.547ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:46019
17042024-08-28T02:15:16.547ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 97 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:46019 req_id = 5e24a230-e941-4909-8730-3b3ea18acded response_code = 200 uri = /vpc-routes
17052024-08-28T02:15:16.556ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 125 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:46019 req_id = 081890a7-5a48-469a-b154-e79b5a0c636d response_code = 204 uri = /vpc-routes
17062024-08-28T02:15:16.624ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:47914
17072024-08-28T02:15:16.624ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 45 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:47914 req_id = c23c1a6b-1200-47d6-8f73-b5104fe262ac response_code = 200 uri = /vpc-routes
17082024-08-28T02:15:16.637ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 101 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:47914 req_id = 3cd1b950-0904-49c8-a628-682dfe735e94 response_code = 204 uri = /vpc-routes
17092024-08-28T02:15:16.670ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:57800
17102024-08-28T02:15:16.670ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 103 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:57800 req_id = 5afddbb9-251c-43b3-a4da-cdb7b357c331 response_code = 200 uri = /v2p
17112024-08-28T02:15:16.829ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:57624
17122024-08-28T02:15:16.830ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 153 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:57624 req_id = e353bb74-2199-4a37-a874-81681fdc7fc0 response_code = 200 uri = /v2p
17132024-08-28T02:15:16.961ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:42362
17142024-08-28T02:15:16.961ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
17152024-08-28T02:15:16.999ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
17162024-08-28T02:15:17.059ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 98086 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:42362 req_id = aed64119-6203-4730-951c-1b3bfe0df9c8 response_code = 204 uri = /switch-ports
17172024-08-28T02:15:17.059ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 115 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:42362 req_id = b50db195-d21c-4df0-bb5f-96a7a7918275 response_code = 200 uri = /network-bootstore-config
17182024-08-28T02:15:17.089ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:35522
17192024-08-28T02:15:17.090ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
17202024-08-28T02:15:17.125ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
17212024-08-28T02:15:17.192ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 102436 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:35522 req_id = bef5ffd6-2b88-4457-b139-47721311dc1e response_code = 204 uri = /switch-ports
17222024-08-28T02:15:17.192ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 155 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:35522 req_id = 4f00f3eb-7e6b-400f-b154-6a2b53886ec4 response_code = 200 uri = /network-bootstore-config
17232024-08-28T02:15:17.361ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:58499
17242024-08-28T02:15:17.361ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 103 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:58499 req_id = bcecd944-9171-494f-9b87-f1b86e962953 response_code = 200 uri = /vpc-routes
17252024-08-28T02:15:17.391ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 140 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:58499 req_id = 401f8b33-86c7-450e-a663-aa56aeec2ed4 response_code = 204 uri = /vpc-routes
17262024-08-28T02:15:17.583ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:58164
17272024-08-28T02:15:17.583ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 159 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:58164 req_id = 4c5c8593-69de-424d-9e50-71f6fbc817bf response_code = 200 uri = /v2p
17282024-08-28T02:15:17.643ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:37538
17292024-08-28T02:15:17.644ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
17302024-08-28T02:15:17.676ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
17312024-08-28T02:15:17.729ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 86120 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:37538 req_id = 8a2eec1c-65b0-4e72-a1f8-6674b0715077 response_code = 204 uri = /switch-ports
17322024-08-28T02:15:17.730ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 96 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:37538 req_id = 608f8767-8ef2-4aa5-9420-6ac6229ebb06 response_code = 200 uri = /network-bootstore-config
17332024-08-28T02:15:19.745ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:42109
17342024-08-28T02:15:19.745ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 113 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:42109 req_id = 7f80515c-779c-47ce-9325-5ddc4721dfb7 response_code = 200 uri = /baseboard
17352024-08-28T02:15:23.456ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 7224 local_addr = [fd00:1122:3344:101::1]:49250 method = GET remote_addr = [fd00:1122:3344:101::d]:45985 req_id = d1553db2-3400-4b2e-bd06-6c1857b21e7a response_code = 200 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261 uri = /40b49400-5cd0-4d5c-a006-b241a1ca8261
17362024-08-28T02:15:46.497ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:42218
17372024-08-28T02:15:46.497ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 108 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:42218 req_id = 1cb78141-fbe7-48bf-be42-bb4013192693 response_code = 200 uri = /vpc-routes
17382024-08-28T02:15:46.523ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:54119
17392024-08-28T02:15:46.523ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 595 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:54119 req_id = 9831a65f-f549-45e8-bd99-16bc10101097 response_code = 200 uri = /vpc-routes
17402024-08-28T02:15:46.527ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:38542
17412024-08-28T02:15:46.527ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 114 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:38542 req_id = 1e3e2faa-38d3-4a3e-bada-dc69e37fd074 response_code = 200 uri = /v2p
17422024-08-28T02:15:46.534ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 105 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:42218 req_id = 3d3b8083-266b-45ca-b16b-b912ee569e02 response_code = 204 uri = /vpc-routes
17432024-08-28T02:15:46.576ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:36642
17442024-08-28T02:15:46.576ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 105 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:36642 req_id = e25a2c79-9bfd-46d1-aaec-2947a8deb2d2 response_code = 200 uri = /v2p
17452024-08-28T02:15:46.580ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 700 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:54119 req_id = 596bf659-6dd8-4edd-b67b-3d8927f053e8 response_code = 204 uri = /vpc-routes
17462024-08-28T02:15:46.601ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:64587
17472024-08-28T02:15:46.602ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
17482024-08-28T02:15:46.639ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
17492024-08-28T02:15:46.696ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 94120 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:64587 req_id = 2ef9675d-9ab5-4ef7-a698-9976551ca456 response_code = 204 uri = /switch-ports
17502024-08-28T02:15:46.696ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 116 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:64587 req_id = 3857582a-3285-4e2b-9c08-6a05d0fdf5bf response_code = 200 uri = /network-bootstore-config
17512024-08-28T02:15:46.738ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:47281
17522024-08-28T02:15:46.738ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
17532024-08-28T02:15:46.775ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
17542024-08-28T02:15:46.829ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 91185 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:47281 req_id = 53609b8f-292f-450a-9b4f-2ca314423867 response_code = 204 uri = /switch-ports
17552024-08-28T02:15:46.829ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 111 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:47281 req_id = d9bd537e-459b-497d-9627-2bd6e238c2db response_code = 200 uri = /network-bootstore-config
17562024-08-28T02:15:47.379ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:51320
17572024-08-28T02:15:47.379ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 110 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:51320 req_id = 2c17e430-773c-4ba8-89fa-005aa941eb5e response_code = 200 uri = /vpc-routes
17582024-08-28T02:15:47.388ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 126 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:51320 req_id = 8c5017b7-6cf1-4c3d-bad3-7428a3bfa09d response_code = 204 uri = /vpc-routes
17592024-08-28T02:15:47.533ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:48095
17602024-08-28T02:15:47.533ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 139 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:48095 req_id = 80949978-cff6-4432-ab46-fc94ad6d8681 response_code = 200 uri = /v2p
17612024-08-28T02:15:47.553ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:45041
17622024-08-28T02:15:47.553ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
17632024-08-28T02:15:47.595ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
17642024-08-28T02:15:47.648ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 95414 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:45041 req_id = 9ff8e5b6-457c-4671-b7a5-651569990c76 response_code = 204 uri = /switch-ports
17652024-08-28T02:15:47.649ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 118 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:45041 req_id = efc21bb8-9a21-4e47-b80c-0bfc0add4ecc response_code = 200 uri = /network-bootstore-config
17662024-08-28T02:15:49.778ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:37343
17672024-08-28T02:15:49.778ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 120 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:37343 req_id = ba1be054-050a-463d-8434-438d1876ee8e response_code = 200 uri = /baseboard
17682024-08-28T02:15:53.455ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 6699 local_addr = [fd00:1122:3344:101::1]:49250 method = GET remote_addr = [fd00:1122:3344:101::d]:45985 req_id = d44c9234-7ffa-43dd-8162-e2cabdfe7bfe response_code = 200 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261 uri = /40b49400-5cd0-4d5c-a006-b241a1ca8261
17692024-08-28T02:16:16.535ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:53625
17702024-08-28T02:16:16.535ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 118 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:53625 req_id = 73bd900e-8947-4825-834d-99f642d01e21 response_code = 200 uri = /vpc-routes
17712024-08-28T02:16:16.552ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 120 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:53625 req_id = bd472dcd-aa00-4b87-a5b3-7a84d08e869d response_code = 204 uri = /vpc-routes
17722024-08-28T02:16:16.707ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:44634
17732024-08-28T02:16:16.707ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 163 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:44634 req_id = 11207005-dff3-4aca-9ac0-43fc32222a79 response_code = 200 uri = /v2p
17742024-08-28T02:16:16.721ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:63760
17752024-08-28T02:16:16.721ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 47 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:63760 req_id = f44e7abd-2f66-40f2-a553-ae30741aa53a response_code = 200 uri = /vpc-routes
17762024-08-28T02:16:16.732ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 102 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:63760 req_id = 406a04f3-3ab9-4292-b8b9-7933705f6adf response_code = 204 uri = /vpc-routes
17772024-08-28T02:16:16.786ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:35127
17782024-08-28T02:16:16.786ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
17792024-08-28T02:16:16.826ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
17802024-08-28T02:16:16.886ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 99843 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:35127 req_id = ecc84583-38fa-44b2-9946-30ce0d366b08 response_code = 204 uri = /switch-ports
17812024-08-28T02:16:16.886ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:63450
17822024-08-28T02:16:16.886ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:46852
17832024-08-28T02:16:16.887ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 239 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:63450 req_id = 54d51c2f-b29e-40b7-b143-a4b32acb79da response_code = 200 uri = /v2p
17842024-08-28T02:16:16.887ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
17852024-08-28T02:16:16.887ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 253 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:35127 req_id = 46397497-b1a6-4629-b14b-67b1a226bf7c response_code = 200 uri = /network-bootstore-config
17862024-08-28T02:16:16.922ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:53439
17872024-08-28T02:16:16.923ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 16f83236-bc4a-4d30-8cf5-b8e748fe9792, kind: Service { id: 6926ac43-4e2e-4378-9bce-42019ec72a30 }, name: Name("external-dns-6926ac43-4e2e-4378-9bce-42019ec72a30"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 235, 144])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 67a6854e-59a4-4c1d-b17a-a4ec8e40b976, kind: Service { id: 7a899aef-896e-41b9-8810-6c5024da5a0c }, name: Name("external-dns-7a899aef-896e-41b9-8810-6c5024da5a0c"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 136, 51])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 138b549d-922d-4976-b2e2-89111b890ac2, kind: Service { id: c359bc38-954c-4b35-9322-26531287be03 }, name: Name("nexus-c359bc38-954c-4b35-9322-26531287be03"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 179, 102])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a81dd239-4fe9-4ce3-b19d-4876383286d2, kind: Service { id: 87cde93b-53d3-4924-847b-c181020478b1 }, name: Name("nexus-87cde93b-53d3-4924-847b-c181020478b1"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 249, 85])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cda1bc8f-c147-42a1-98e0-40955e79f4cf, kind: Service { id: 50ac3e36-ea5c-4742-b23f-aa0a78804739 }, name: Name("nexus-50ac3e36-ea5c-4742-b23f-aa0a78804739"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 180])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
17882024-08-28T02:16:16.923ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17892024-08-28T02:16:16.923ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
17902024-08-28T02:16:16.923ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
17912024-08-28T02:16:16.923ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17922024-08-28T02:16:16.923ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17932024-08-28T02:16:16.923ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
17942024-08-28T02:16:16.923ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1092 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:53439 req_id = 6aaf3b71-411c-4bb8-bdc6-8061075f10c0 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
17952024-08-28T02:16:16.924ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
17962024-08-28T02:16:16.951ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:54708
17972024-08-28T02:16:16.951ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 16f83236-bc4a-4d30-8cf5-b8e748fe9792, kind: Service { id: 6926ac43-4e2e-4378-9bce-42019ec72a30 }, name: Name("external-dns-6926ac43-4e2e-4378-9bce-42019ec72a30"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 235, 144])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 67a6854e-59a4-4c1d-b17a-a4ec8e40b976, kind: Service { id: 7a899aef-896e-41b9-8810-6c5024da5a0c }, name: Name("external-dns-7a899aef-896e-41b9-8810-6c5024da5a0c"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 136, 51])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 138b549d-922d-4976-b2e2-89111b890ac2, kind: Service { id: c359bc38-954c-4b35-9322-26531287be03 }, name: Name("nexus-c359bc38-954c-4b35-9322-26531287be03"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 179, 102])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a81dd239-4fe9-4ce3-b19d-4876383286d2, kind: Service { id: 87cde93b-53d3-4924-847b-c181020478b1 }, name: Name("nexus-87cde93b-53d3-4924-847b-c181020478b1"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 249, 85])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cda1bc8f-c147-42a1-98e0-40955e79f4cf, kind: Service { id: 50ac3e36-ea5c-4742-b23f-aa0a78804739 }, name: Name("nexus-50ac3e36-ea5c-4742-b23f-aa0a78804739"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 180])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
17982024-08-28T02:16:16.951ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17992024-08-28T02:16:16.952ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
18002024-08-28T02:16:16.952ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
18012024-08-28T02:16:16.952ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18022024-08-28T02:16:16.952ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18032024-08-28T02:16:16.952ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
18042024-08-28T02:16:16.952ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 652 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:54708 req_id = 5f76130c-8f37-4f4b-aec1-f668c8722713 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
18052024-08-28T02:16:16.980ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 93085 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:46852 req_id = 53fae56e-ff62-4212-8f85-6e9da9699f1c response_code = 204 uri = /switch-ports
18062024-08-28T02:16:16.980ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 123 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:46852 req_id = 5115e53f-753b-4843-9cf1-b2e44b34fefb response_code = 200 uri = /network-bootstore-config
18072024-08-28T02:16:17.413ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:44376
18082024-08-28T02:16:17.414ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 77 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:44376 req_id = 6ac8f435-d994-4ce8-a68d-800d72823e77 response_code = 200 uri = /vpc-routes
18092024-08-28T02:16:17.424ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 113 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:44376 req_id = 3e20685e-2107-4e3f-9f3f-ec647a402621 response_code = 204 uri = /vpc-routes
18102024-08-28T02:16:17.673ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:35227
18112024-08-28T02:16:17.674ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 808 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:35227 req_id = 514a1185-86be-467a-9992-21a7d87a9e30 response_code = 200 uri = /v2p
18122024-08-28T02:16:17.674ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:59187
18132024-08-28T02:16:17.674ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
18142024-08-28T02:16:17.713ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
18152024-08-28T02:16:17.766ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 91780 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:59187 req_id = 6d161c63-582a-4bcf-82de-94140e9bdd1f response_code = 204 uri = /switch-ports
18162024-08-28T02:16:17.766ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 151 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:59187 req_id = a88073e8-980a-4071-8d92-e9ad3e76ef68 response_code = 200 uri = /network-bootstore-config
18172024-08-28T02:16:17.777ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:49958
18182024-08-28T02:16:17.777ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:671 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 16f83236-bc4a-4d30-8cf5-b8e748fe9792, kind: Service { id: 6926ac43-4e2e-4378-9bce-42019ec72a30 }, name: Name("external-dns-6926ac43-4e2e-4378-9bce-42019ec72a30"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 235, 144])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 67a6854e-59a4-4c1d-b17a-a4ec8e40b976, kind: Service { id: 7a899aef-896e-41b9-8810-6c5024da5a0c }, name: Name("external-dns-7a899aef-896e-41b9-8810-6c5024da5a0c"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 136, 51])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 138b549d-922d-4976-b2e2-89111b890ac2, kind: Service { id: c359bc38-954c-4b35-9322-26531287be03 }, name: Name("nexus-c359bc38-954c-4b35-9322-26531287be03"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 179, 102])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a81dd239-4fe9-4ce3-b19d-4876383286d2, kind: Service { id: 87cde93b-53d3-4924-847b-c181020478b1 }, name: Name("nexus-87cde93b-53d3-4924-847b-c181020478b1"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 249, 85])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cda1bc8f-c147-42a1-98e0-40955e79f4cf, kind: Service { id: 50ac3e36-ea5c-4742-b23f-aa0a78804739 }, name: Name("nexus-50ac3e36-ea5c-4742-b23f-aa0a78804739"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 180])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
18192024-08-28T02:16:17.777ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18202024-08-28T02:16:17.777ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
18212024-08-28T02:16:17.777ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
18222024-08-28T02:16:17.777ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18232024-08-28T02:16:17.777ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18242024-08-28T02:16:17.777ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:690 port = "opte0" rules = []
18252024-08-28T02:16:17.777ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 426 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:49958 req_id = 854c0ce3-87dd-4f2c-b49b-724837dd3db3 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
18262024-08-28T02:16:19.811ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:52248
18272024-08-28T02:16:19.811ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 94 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:52248 req_id = ebb49814-fcd8-4a82-87ae-9d0ce348a751 response_code = 200 uri = /baseboard
18282024-08-28T02:16:23.454ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 4921 local_addr = [fd00:1122:3344:101::1]:49250 method = GET remote_addr = [fd00:1122:3344:101::d]:45985 req_id = 5a80ae15-306b-4482-b22e-a5d23f2c5505 response_code = 200 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261 uri = /40b49400-5cd0-4d5c-a006-b241a1ca8261
18292024-08-28T02:16:46.519ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:46619
18302024-08-28T02:16:46.520ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 950 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:46619 req_id = 9ee0d24d-7fde-4344-b07d-9d43c0e4856e response_code = 200 uri = /vpc-routes
18312024-08-28T02:16:46.532ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 123 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:46619 req_id = 68bad2f1-7900-46a1-a2e8-026bf8548d42 response_code = 204 uri = /vpc-routes
18322024-08-28T02:16:46.537ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:52080
18332024-08-28T02:16:46.537ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 38 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:52080 req_id = cb9398af-4347-4e20-95d6-ab86fc848e86 response_code = 200 uri = /vpc-routes
18342024-08-28T02:16:46.547ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 83 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:52080 req_id = a03b5d28-15b6-4073-aa2d-fc69f91b6740 response_code = 204 uri = /vpc-routes
18352024-08-28T02:16:46.599ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:55877
18362024-08-28T02:16:46.599ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 121 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:55877 req_id = 77751ea0-0528-40c0-a67c-af5445ae4e53 response_code = 200 uri = /v2p
18372024-08-28T02:16:46.611ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:56082
18382024-08-28T02:16:46.611ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
18392024-08-28T02:16:46.650ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
18402024-08-28T02:16:46.713ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 102102 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:56082 req_id = 8ef0b249-ceab-4000-8fe1-276d161c2977 response_code = 204 uri = /switch-ports
18412024-08-28T02:16:46.714ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:63948
18422024-08-28T02:16:46.714ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:48210
18432024-08-28T02:16:46.714ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
18442024-08-28T02:16:46.715ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 900 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:48210 req_id = 38aee3d9-564d-44b8-9694-3b240b3c1e01 response_code = 200 uri = /v2p
18452024-08-28T02:16:46.718ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 4463 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:56082 req_id = f310bab9-2edf-4af6-b850-54c37cff8cd0 response_code = 200 uri = /network-bootstore-config
18462024-08-28T02:16:46.750ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
18472024-08-28T02:16:46.806ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 92239 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:63948 req_id = 3354afd0-8c38-49d6-b818-5bfdddc62562 response_code = 204 uri = /switch-ports
18482024-08-28T02:16:46.807ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 747 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:63948 req_id = e88ec0c9-43f9-40be-904a-a88c6954ff75 response_code = 200 uri = /network-bootstore-config
18492024-08-28T02:16:47.597ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:55470
18502024-08-28T02:16:47.598ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 110 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:55470 req_id = 78ea0a0b-ab22-465d-a05d-26bf68cc223d response_code = 200 uri = /vpc-routes
18512024-08-28T02:16:47.607ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 125 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:55470 req_id = a113414a-3401-4224-8123-125c24f2d40f response_code = 204 uri = /vpc-routes
18522024-08-28T02:16:47.671ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:52687
18532024-08-28T02:16:47.671ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 139 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:52687 req_id = f379582c-ff8c-4166-92bb-16c5c8d29929 response_code = 200 uri = /v2p
18542024-08-28T02:16:47.728ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:63964
18552024-08-28T02:16:47.728ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
18562024-08-28T02:16:47.768ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
18572024-08-28T02:16:47.827ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 98930 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:63964 req_id = abab3582-edbd-4d8f-91cd-1051fc0a3b3c response_code = 204 uri = /switch-ports
18582024-08-28T02:16:47.828ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 107 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:63964 req_id = 525ade51-52b5-4315-a07f-12c374db9f85 response_code = 200 uri = /network-bootstore-config
18592024-08-28T02:16:49.844ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:55970
18602024-08-28T02:16:49.844ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 92 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:55970 req_id = 1bdec283-b579-4521-9f01-566e425aea6d response_code = 200 uri = /baseboard
18612024-08-28T02:16:53.514ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 5817 local_addr = [fd00:1122:3344:101::1]:49250 method = GET remote_addr = [fd00:1122:3344:101::d]:45985 req_id = 55ab365f-a160-40d4-b311-cef11d6f5f00 response_code = 200 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261 uri = /40b49400-5cd0-4d5c-a006-b241a1ca8261
18622024-08-28T02:17:16.554ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:50142
18632024-08-28T02:17:16.554ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 101 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:50142 req_id = 8d86783a-1f56-4e1b-b853-3e0b364e3622 response_code = 200 uri = /vpc-routes
18642024-08-28T02:17:16.574ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 175 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:50142 req_id = 0fd17bb5-e8ac-4915-8223-794d32018534 response_code = 204 uri = /vpc-routes
18652024-08-28T02:17:16.676ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:59623
18662024-08-28T02:17:16.676ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 55 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:59623 req_id = 9528b8bd-8135-4978-a4ef-28192fb87d7a response_code = 200 uri = /vpc-routes
18672024-08-28T02:17:16.686ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:50445
18682024-08-28T02:17:16.687ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1232 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:50445 req_id = 5447833d-ccef-48a1-b749-c7b3a47039dc response_code = 200 uri = /v2p
18692024-08-28T02:17:16.695ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 87 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:59623 req_id = 6719f113-5d97-4286-a66a-9da0d0c06b21 response_code = 204 uri = /vpc-routes
18702024-08-28T02:17:16.778ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:61479
18712024-08-28T02:17:16.778ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 96 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:61479 req_id = 36436965-2647-4fa2-ae79-a12c951609c4 response_code = 200 uri = /v2p
18722024-08-28T02:17:16.819ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:42071
18732024-08-28T02:17:16.819ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
18742024-08-28T02:17:16.855ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
18752024-08-28T02:17:16.909ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 90696 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:42071 req_id = f404dde3-4ebd-4f9a-a0f0-b3040578d318 response_code = 204 uri = /switch-ports
18762024-08-28T02:17:16.910ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 225 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:42071 req_id = 3ceffe25-6167-4309-a8d3-33363a32ac98 response_code = 200 uri = /network-bootstore-config
18772024-08-28T02:17:16.941ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:42144
18782024-08-28T02:17:16.941ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
18792024-08-28T02:17:16.975ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
18802024-08-28T02:17:17.035ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 94031 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:42144 req_id = f3e0240a-c9b4-4eac-be60-4afb554ed812 response_code = 204 uri = /switch-ports
18812024-08-28T02:17:17.035ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 116 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:42144 req_id = a426d2c2-ced0-42d2-a138-8ab946123125 response_code = 200 uri = /network-bootstore-config
18822024-08-28T02:17:17.625ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:39081
18832024-08-28T02:17:17.626ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1025 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:39081 req_id = c2fac908-75ed-4db8-9a02-afdfdae7c238 response_code = 200 uri = /v2p
18842024-08-28T02:17:17.662ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:56809
18852024-08-28T02:17:17.662ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 74 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:56809 req_id = 047806d3-7aaa-4444-9abb-aea15c4ff436 response_code = 200 uri = /vpc-routes
18862024-08-28T02:17:17.672ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 134 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:56809 req_id = 81b8acb3-96dc-4260-9383-9f2d04574e26 response_code = 204 uri = /vpc-routes
18872024-08-28T02:17:17.739ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:46980
18882024-08-28T02:17:17.740ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
18892024-08-28T02:17:17.776ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
18902024-08-28T02:17:17.830ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 90320 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:46980 req_id = e8769c4d-6cb3-4161-8b3f-8e92d187c5a4 response_code = 204 uri = /switch-ports
18912024-08-28T02:17:17.830ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 105 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:46980 req_id = 821c0707-754a-4864-b3a6-537a358c014e response_code = 200 uri = /network-bootstore-config
18922024-08-28T02:17:19.879ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:64295
18932024-08-28T02:17:19.879ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 91 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:64295 req_id = 4c7721b3-ad9e-4019-85c9-1a8e99b45b12 response_code = 200 uri = /baseboard
18942024-08-28T02:17:23.515ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 5651 local_addr = [fd00:1122:3344:101::1]:49250 method = GET remote_addr = [fd00:1122:3344:101::d]:45985 req_id = c7e3cd45-390e-499a-9500-e278c9f69124 response_code = 200 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261 uri = /40b49400-5cd0-4d5c-a006-b241a1ca8261
18952024-08-28T02:17:44.207ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:63792
18962024-08-28T02:17:44.207ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 120 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:63792 req_id = 3d8a6c98-e926-443a-9826-15f3c5c42ee5 response_code = 200 uri = /v2p
18972024-08-28T02:17:44.208ZINFOSledAgent (PortManager): Mapping virtual NIC to physical host file = illumos-utils/src/opte/port_manager.rs:778 mapping = VirtualNetworkInterfaceHost { virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 243, 119, 22])), physical_host_ip: fd00:1122:3344:101::1, vni: Vni(6178120) }
18982024-08-28T02:17:44.208ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 603 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:63792 req_id = 38d0fe0c-904e-41f7-af07-598bb76d219a response_code = 204 uri = /v2p
18992024-08-28T02:17:44.398ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:34741
19002024-08-28T02:17:44.407ZINFOSledAgent (InstanceManager): ensuring instance is registered file = sled-agent/src/instance_manager.rs:570 hardware = InstanceHardware { properties: InstanceProperties { ncpus: InstanceCpuCount(2), memory: ByteCount(1073741824), hostname: Hostname("localshark") }, nics: [NetworkInterface { id: fd26a776-1dfe-477f-bf08-a96ab45447db, kind: Instance { id: 29444af2-b828-4c13-9cb7-17c7a56ed0e4 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 243, 119, 22])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(6178120), primary: true, slot: 0, transit_ips: [] }], source_nat: SourceNatConfig { ip: 10.151.2.175, first_port: 0, last_port: 16383 }, ephemeral_ip: Some(10.151.2.176), floating_ips: [], firewall_rules: [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: fd26a776-1dfe-477f-bf08-a96ab45447db, kind: Instance { id: 29444af2-b828-4c13-9cb7-17c7a56ed0e4 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 243, 119, 22])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(6178120), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: fd26a776-1dfe-477f-bf08-a96ab45447db, kind: Instance { id: 29444af2-b828-4c13-9cb7-17c7a56ed0e4 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 243, 119, 22])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(6178120), primary: true, slot: 0, transit_ips: [] }], filter_hosts: Some([Vpc(Vni(6178120))]), filter_ports: None, filter_protocols: None, action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: fd26a776-1dfe-477f-bf08-a96ab45447db, kind: Instance { id: 29444af2-b828-4c13-9cb7-17c7a56ed0e4 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 243, 119, 22])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(6178120), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(22), last: L4Port(22) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }], dhcp_config: DhcpConfig { dns_servers: [1.1.1.1, 9.9.9.9], host_domain: None, search_domains: [] }, disks: [DiskRequest { device: "nvme", name: "disk-e01175c6c07", read_only: false, slot: Slot(0), volume_construction_request: Volume { block_size: 512, id: 942302a9-1463-425c-9b9b-7a3c5b829867, read_only_parent: Some(Volume { block_size: 512, id: 54862bcf-81f4-4404-b78d-1e00dd8c974e, read_only_parent: None, sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 2, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: c8cc2e90-f729-40fe-9756-becda66e3a7e, key: Some("qIchoWWc2ZRhdoK5mzCyuOC1mGQM5wCca/tOyHnu8/0="), key_pem: None, lossy: false, read_only: true, root_cert_pem: None, target: ["[fd00:1122:3344:101::12]:19001", "[fd00:1122:3344:101::1a]:19001", "[fd00:1122:3344:101::17]:19001"] } }] }), sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 1, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 942302a9-1463-425c-9b9b-7a3c5b829867, key: Some("S+nooGSMj55h0v2fBUfXH7qfGCWOuSOheKJ6AzSwnxM="), key_pem: None, lossy: false, read_only: false, root_cert_pem: None, target: ["[fd00:1122:3344:101::15]:19001", "[fd00:1122:3344:101::13]:19001", "[fd00:1122:3344:101::19]:19000"] } }] } }], cloud_init_bytes: Some(..) } instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 instance_runtime = InstanceRuntimeState { propolis_id: Some(8b43f99c-a0e7-445b-8ea1-1ee3b968f594 (propolis)), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2024-08-28T02:17:43.969444548Z } metadata = InstanceMetadata { silo_id: bb0007f2-a695-4aac-b57b-8cca99a33c5b, project_id: 286513bd-9597-457f-90e3-0d7965df272f } propolis_addr = [fd00:1122:3344:101::1:0]:12400 propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594 vmm_runtime = VmmRuntimeState { state: Starting, gen: Generation(1), time_updated: 2024-08-28T02:17:43.923233Z }
19012024-08-28T02:17:44.407ZINFOSledAgent (InstanceManager): registering new instance file = sled-agent/src/instance_manager.rs:603 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594
19022024-08-28T02:17:44.407ZINFOSledAgent (InstanceManager): initializing new Instance file = sled-agent/src/instance.rs:991 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 migration_id = None propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594 state = InstanceInitialState { hardware: InstanceHardware { properties: InstanceProperties { ncpus: InstanceCpuCount(2), memory: ByteCount(1073741824), hostname: Hostname("localshark") }, nics: [NetworkInterface { id: fd26a776-1dfe-477f-bf08-a96ab45447db, kind: Instance { id: 29444af2-b828-4c13-9cb7-17c7a56ed0e4 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 243, 119, 22])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(6178120), primary: true, slot: 0, transit_ips: [] }], source_nat: SourceNatConfig { ip: 10.151.2.175, first_port: 0, last_port: 16383 }, ephemeral_ip: Some(10.151.2.176), floating_ips: [], firewall_rules: [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: fd26a776-1dfe-477f-bf08-a96ab45447db, kind: Instance { id: 29444af2-b828-4c13-9cb7-17c7a56ed0e4 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 243, 119, 22])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(6178120), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: fd26a776-1dfe-477f-bf08-a96ab45447db, kind: Instance { id: 29444af2-b828-4c13-9cb7-17c7a56ed0e4 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 243, 119, 22])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(6178120), primary: true, slot: 0, transit_ips: [] }], filter_hosts: Some([Vpc(Vni(6178120))]), filter_ports: None, filter_protocols: None, action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: fd26a776-1dfe-477f-bf08-a96ab45447db, kind: Instance { id: 29444af2-b828-4c13-9cb7-17c7a56ed0e4 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 243, 119, 22])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(6178120), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(22), last: L4Port(22) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }], dhcp_config: DhcpConfig { dns_servers: [1.1.1.1, 9.9.9.9], host_domain: None, search_domains: [] }, disks: [DiskRequest { device: "nvme", name: "disk-e01175c6c07", read_only: false, slot: Slot(0), volume_construction_request: Volume { block_size: 512, id: 942302a9-1463-425c-9b9b-7a3c5b829867, read_only_parent: Some(Volume { block_size: 512, id: 54862bcf-81f4-4404-b78d-1e00dd8c974e, read_only_parent: None, sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 2, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: c8cc2e90-f729-40fe-9756-becda66e3a7e, key: Some("qIchoWWc2ZRhdoK5mzCyuOC1mGQM5wCca/tOyHnu8/0="), key_pem: None, lossy: false, read_only: true, root_cert_pem: None, target: ["[fd00:1122:3344:101::12]:19001", "[fd00:1122:3344:101::1a]:19001", "[fd00:1122:3344:101::17]:19001"] } }] }), sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 1, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 942302a9-1463-425c-9b9b-7a3c5b829867, key: Some("S+nooGSMj55h0v2fBUfXH7qfGCWOuSOheKJ6AzSwnxM="), key_pem: None, lossy: false, read_only: false, root_cert_pem: None, target: ["[fd00:1122:3344:101::15]:19001", "[fd00:1122:3344:101::13]:19001", "[fd00:1122:3344:101::19]:19000"] } }] } }], cloud_init_bytes: Some(..) }, vmm_runtime: VmmRuntimeState { state: Starting, gen: Generation(1), time_updated: 2024-08-28T02:17:43.923233Z }, propolis_addr: [fd00:1122:3344:101::1:0]:12400, migration_id: None }
19032024-08-28T02:17:44.409ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 10770 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:34741 req_id = 7da75c57-2b53-4fce-b880-e8a98288e067 response_code = 200 uri = /vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594
19042024-08-28T02:17:44.511ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:63875
19052024-08-28T02:17:44.513ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1823 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:63875 req_id = 0c7fa2e5-ca19-45a9-bd0a-faf56cd8fa0b response_code = 200 uri = /vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594/state
19062024-08-28T02:17:44.514ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:424 port = Port { inner: PortInner(PortData { name: "opte6", ip: 172.30.0.5, mac: MacAddr6([168, 64, 37, 243, 119, 22]), slot: 0, vni: Vni { inner: 6178120 }, subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.0.1 } }) }
19072024-08-28T02:17:44.516ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
19082024-08-28T02:17:44.554ZINFOSledAgent (InstanceManager): Configuring new Omicron zone: oxz_propolis-server_8b43f99c-a0e7-445b-8ea1-1ee3b968f594 file = illumos-utils/src/zone.rs:316 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594
19092024-08-28T02:17:44.578ZINFOSledAgent (InstanceManager): Installing Omicron zone: oxz_propolis-server_8b43f99c-a0e7-445b-8ea1-1ee3b968f594 file = illumos-utils/src/zone.rs:354 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594
19102024-08-28T02:17:45.774ZINFOSledAgent (InstanceManager): Profile for oxz_propolis-server_8b43f99c-a0e7-445b-8ea1-1ee3b968f594: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="system/illumos/propolis-server"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlInstance0'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::1:0'/> <propval type="astring" name="listen_port" value='12400'/> <propval type="astring" name="metric_addr" value='dns'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594
19112024-08-28T02:17:45.774ZINFOSledAgent (InstanceManager): Booting oxz_propolis-server_8b43f99c-a0e7-445b-8ea1-1ee3b968f594 zone file = illumos-utils/src/running_zone.rs:488 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594 zone = oxz_propolis-server_8b43f99c-a0e7-445b-8ea1-1ee3b968f594
19122024-08-28T02:17:45.962ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_8b43f99c-a0e7-445b-8ea1-1ee3b968f594") failed: Property not found. retry in 49.405691ms file = illumos-utils/src/svc.rs:36 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594 zone = oxz_propolis-server_8b43f99c-a0e7-445b-8ea1-1ee3b968f594
19132024-08-28T02:17:46.022ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_8b43f99c-a0e7-445b-8ea1-1ee3b968f594") failed: Property not found. retry in 56.772968ms file = illumos-utils/src/svc.rs:36 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594 zone = oxz_propolis-server_8b43f99c-a0e7-445b-8ea1-1ee3b968f594
19142024-08-28T02:17:46.094ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_8b43f99c-a0e7-445b-8ea1-1ee3b968f594") failed: Property not found. retry in 141.715393ms file = illumos-utils/src/svc.rs:36 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594 zone = oxz_propolis-server_8b43f99c-a0e7-445b-8ea1-1ee3b968f594
19152024-08-28T02:17:46.251ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_8b43f99c-a0e7-445b-8ea1-1ee3b968f594") failed: Property not found. retry in 464.232574ms file = illumos-utils/src/svc.rs:36 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594 zone = oxz_propolis-server_8b43f99c-a0e7-445b-8ea1-1ee3b968f594
19162024-08-28T02:17:46.474ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:50300
19172024-08-28T02:17:46.488ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:43742
19182024-08-28T02:17:46.532ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:52859
19192024-08-28T02:17:46.532ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 76 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:52859 req_id = 9fad52fa-9aba-49d7-9c08-33cb5ad8fbea response_code = 200 uri = /vpc-routes
19202024-08-28T02:17:46.565ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:40274
19212024-08-28T02:17:46.565ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 250 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:40274 req_id = 975bb633-fe9c-4bea-a4b5-d041e38e1a8f response_code = 200 uri = /v2p
19222024-08-28T02:17:46.627ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 288 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:52859 req_id = 46c62797-2c8c-4825-8976-d5eef3754768 response_code = 204 uri = /vpc-routes
19232024-08-28T02:17:46.633ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51763
19242024-08-28T02:17:46.633ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 40 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:51763 req_id = 90ae09fb-a510-4015-aff5-e876d443dacf response_code = 200 uri = /vpc-routes
19252024-08-28T02:17:46.661ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1091 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:51763 req_id = ae383cac-7932-40ef-8511-1b6412edef28 response_code = 204 uri = /vpc-routes
19262024-08-28T02:17:46.670ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:39470
19272024-08-28T02:17:46.670ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
19282024-08-28T02:17:46.716ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
19292024-08-28T02:17:46.789ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 118866 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:39470 req_id = 834b7f46-26c9-4464-aedd-945f1a942561 response_code = 204 uri = /switch-ports
19302024-08-28T02:17:46.794ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:34803
19312024-08-28T02:17:46.794ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 483 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:34803 req_id = dfdae915-295a-40c0-8bd8-3b1ad6f91302 response_code = 200 uri = /v2p
19322024-08-28T02:17:46.794ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1438 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:39470 req_id = ac894731-daf1-401b-bad1-32f4a89d9518 response_code = 200 uri = /network-bootstore-config
19332024-08-28T02:17:46.800ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_8b43f99c-a0e7-445b-8ea1-1ee3b968f594") failed: Property not found. retry in 855.63942ms file = illumos-utils/src/svc.rs:36 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594 zone = oxz_propolis-server_8b43f99c-a0e7-445b-8ea1-1ee3b968f594
19342024-08-28T02:17:46.820ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:56220
19352024-08-28T02:17:46.820ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
19362024-08-28T02:17:46.867ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
19372024-08-28T02:17:46.942ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 121966 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:56220 req_id = 89ae33de-88a9-4618-8e1f-530ee3b74cb4 response_code = 204 uri = /switch-ports
19382024-08-28T02:17:46.942ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 118 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:56220 req_id = 8983982c-a7f7-4179-a7d2-151a0563a76f response_code = 200 uri = /network-bootstore-config
19392024-08-28T02:17:47.354ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:49902
19402024-08-28T02:17:47.354ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 107 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:49902 req_id = 567c00a4-22a1-4721-8f78-5ac8a1ee3438 response_code = 200 uri = /vpc-routes
19412024-08-28T02:17:47.395ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:43660
19422024-08-28T02:17:47.486ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:42791
19432024-08-28T02:17:47.487ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 822 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:42791 req_id = 3aee917a-5e1d-41f6-a033-44f971309470 response_code = 200 uri = /v2p
19442024-08-28T02:17:47.494ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 111 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:49902 req_id = c561d007-bc03-466a-9ab8-dfb69f18dcee response_code = 204 uri = /vpc-routes
19452024-08-28T02:17:47.624ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:50617
19462024-08-28T02:17:47.625ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4033
19472024-08-28T02:17:47.662ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4057
19482024-08-28T02:17:47.721ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 96117 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:50617 req_id = 7d0bda70-c070-4082-90df-61340755e9d7 response_code = 204 uri = /switch-ports
19492024-08-28T02:17:47.725ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 747 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:50617 req_id = 42ada670-d85f-4328-97ac-7679d29ecf41 response_code = 200 uri = /network-bootstore-config
19502024-08-28T02:17:47.732ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_8b43f99c-a0e7-445b-8ea1-1ee3b968f594") failed: Property not found. retry in 743.545795ms file = illumos-utils/src/svc.rs:36 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594 zone = oxz_propolis-server_8b43f99c-a0e7-445b-8ea1-1ee3b968f594
19512024-08-28T02:17:48.487ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_8b43f99c-a0e7-445b-8ea1-1ee3b968f594") failed: Property not found. retry in 1.414403071s file = illumos-utils/src/svc.rs:36 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594 zone = oxz_propolis-server_8b43f99c-a0e7-445b-8ea1-1ee3b968f594
19522024-08-28T02:17:49.913ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_8b43f99c-a0e7-445b-8ea1-1ee3b968f594") failed: Property not found. retry in 1.001123981s file = illumos-utils/src/svc.rs:36 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594 zone = oxz_propolis-server_8b43f99c-a0e7-445b-8ea1-1ee3b968f594
19532024-08-28T02:17:49.913ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:54583
19542024-08-28T02:17:49.913ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 82 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:54583 req_id = bfd98180-e48f-4b48-8b6e-7da74c37fc2b response_code = 200 uri = /baseboard
19552024-08-28T02:17:50.946ZINFOSledAgent (InstanceManager): Started propolis in zone: oxz_propolis-server_8b43f99c-a0e7-445b-8ea1-1ee3b968f594 file = sled-agent/src/instance.rs:1449 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594
19562024-08-28T02:17:50.956ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_8b43f99c-a0e7-445b-8ea1-1ee3b968f594") failed: Property not found. retry in 35.755424ms file = illumos-utils/src/svc.rs:36 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594
19572024-08-28T02:17:51.003ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_8b43f99c-a0e7-445b-8ea1-1ee3b968f594") failed: Property not found. retry in 61.945172ms file = illumos-utils/src/svc.rs:36 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594
19582024-08-28T02:17:51.076ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_8b43f99c-a0e7-445b-8ea1-1ee3b968f594") failed: Property not found. retry in 102.604484ms file = illumos-utils/src/svc.rs:36 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594
19592024-08-28T02:17:51.188ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_8b43f99c-a0e7-445b-8ea1-1ee3b968f594") failed: Property not found. retry in 318.303016ms file = illumos-utils/src/svc.rs:36 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594
19602024-08-28T02:17:51.516ZINFOSledAgent (InstanceManager): Propolis SMF service is online file = sled-agent/src/instance.rs:1458 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594
19612024-08-28T02:17:51.555ZINFOSledAgent (InstanceManager): Propolis HTTP server online file = sled-agent/src/instance.rs:1482 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594
19622024-08-28T02:17:51.696ZINFOSledAgent (InstanceManager): result of instance_ensure call is Ok(InstanceEnsureResponse { migrate: None }) file = sled-agent/src/instance.rs:727 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594
19632024-08-28T02:17:51.697ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 4302143 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:43660 req_id = 628daf21-533a-4de6-97a8-a7a705fdd40c response_code = 200 uri = /vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594/state
19642024-08-28T02:17:51.697ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 5208898 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:43742 req_id = 1ddcd5b4-a0f6-4053-885b-7bb91de7b7d7 response_code = 200 uri = /vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594/state
19652024-08-28T02:17:51.697ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 5222384 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:50300 req_id = b0939963-93b5-4898-9ff6-e46089116db7 response_code = 200 uri = /vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594/state
19662024-08-28T02:17:51.698ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Creating), migration_in: None, migration_out: None, time: 2024-08-28T02:17:51.698662477Z } file = sled-agent/src/instance.rs:594 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594
19672024-08-28T02:17:51.698ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:616 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 new_vmm_state = VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2024-08-28T02:17:51.698662477Z } propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594
19682024-08-28T02:17:51.698ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:514 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594 state = SledVmmState { vmm_state: VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2024-08-28T02:17:51.698662477Z }, migration_in: None, migration_out: None }
19692024-08-28T02:17:51.746ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Running), migration_in: None, migration_out: None, time: 2024-08-28T02:17:51.746880984Z } file = sled-agent/src/instance.rs:594 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594
19702024-08-28T02:17:51.746ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:616 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 new_vmm_state = VmmRuntimeState { state: Running, gen: Generation(3), time_updated: 2024-08-28T02:17:51.746880984Z } propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594
19712024-08-28T02:17:51.746ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:514 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594 state = SledVmmState { vmm_state: VmmRuntimeState { state: Running, gen: Generation(3), time_updated: 2024-08-28T02:17:51.746880984Z }, migration_in: None, migration_out: None }
19722024-08-28T02:17:53.516ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 6716 local_addr = [fd00:1122:3344:101::1]:49250 method = GET remote_addr = [fd00:1122:3344:101::d]:45985 req_id = a533acce-a46c-4cd3-a733-bb0a556e8283 response_code = 200 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261 uri = /40b49400-5cd0-4d5c-a006-b241a1ca8261
19732024-08-28T02:18:02.036ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:62156
19742024-08-28T02:18:02.036ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 633 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:62156 req_id = f1e36851-f361-4f0e-8307-090e13a9fdf8 response_code = 200 uri = /vmms/8b43f99c-a0e7-445b-8ea1-1ee3b968f594/state
19752024-08-28T02:18:02.036ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Stopping), migration_in: None, migration_out: None, time: 2024-08-28T02:18:02.036931604Z } file = sled-agent/src/instance.rs:594 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594
19762024-08-28T02:18:02.036ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:616 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 new_vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(5), time_updated: 2024-08-28T02:18:02.036931604Z } propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594
19772024-08-28T02:18:02.036ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:514 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594 state = SledVmmState { vmm_state: VmmRuntimeState { state: Stopping, gen: Generation(5), time_updated: 2024-08-28T02:18:02.036931604Z }, migration_in: None, migration_out: None }
19782024-08-28T02:18:02.047ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Destroyed), migration_in: None, migration_out: None, time: 2024-08-28T02:18:02.047117116Z } file = sled-agent/src/instance.rs:594 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594
19792024-08-28T02:18:02.047ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:616 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 new_vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(6), time_updated: 2024-08-28T02:18:02.047117116Z } propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594
19802024-08-28T02:18:02.047ZINFOSledAgent (InstanceManager): terminating VMM that has exited file = sled-agent/src/instance.rs:630 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594
19812024-08-28T02:18:02.047ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
19822024-08-28T02:18:02.047ZWARNSledAgent (kstat-sampler-worker): target removed with queued samples file = oximeter/instruments/src/kstat/sampler.rs:557 id = 11742096108725721107 n_samples = 6 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261
19832024-08-28T02:18:02.047ZWARNSledAgent (kstat-sampler-worker): target removed with queued samples file = oximeter/instruments/src/kstat/sampler.rs:557 id = 5068807404987314413 n_samples = 6 sled_id = 40b49400-5cd0-4d5c-a006-b241a1ca8261
19842024-08-28T02:18:02.048ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:406
19852024-08-28T02:18:02.048ZINFOSledAgent (ZoneBundler): creating zone bundle context = ZoneBundleContext { storage_dirs: ["/pool/int/3dd1ecc3-6cd1-48fe-8e10-622a58ff4257/debug/bundle/zone", "/pool/int/fadb702a-2570-430e-a63c-e4ed78dd68d2/debug/bundle/zone"], cause: TerminatedInstance, extra_log_dirs: ["/pool/ext/fe881465-8582-4eba-8c48-1f5016ef66f5/crypt/debug", "/pool/ext/79e9b421-598b-4117-bcfb-c4abc62898c1/crypt/debug", "/pool/ext/ae881951-d9c0-4515-a6b0-7e5c2297d58f/crypt/debug", "/pool/ext/33490a89-0fb3-432e-bbc7-3c605249772c/crypt/debug", "/pool/ext/9b46afde-44d9-45f0-a9fc-f4fa353c3113/crypt/debug", "/pool/ext/5d8fb525-afb0-4464-ad9b-8e50ab9d715a/crypt/debug", "/pool/ext/95dac2cb-734b-49b1-8778-a286d737d1fa/crypt/debug", "/pool/ext/37700612-da7c-4ef5-9910-6b71dd793192/crypt/debug", "/pool/ext/e56ec7bf-ebda-425c-b581-1265bb74473b/crypt/debug"] } file = sled-agent/src/zone_bundle.rs:339 zone_name = oxz_propolis-server_8b43f99c-a0e7-445b-8ea1-1ee3b968f594
19862024-08-28T02:18:02.672ZINFOSledAgent (ZoneBundler): finished zone bundle file = sled-agent/src/zone_bundle.rs:1137 metadata = ZoneBundleMetadata { id: ZoneBundleId { zone_name: "oxz_propolis-server_8b43f99c-a0e7-445b-8ea1-1ee3b968f594", bundle_id: 3671dd0b-7192-4920-838c-70f21e2a051b }, time_created: 2024-08-28T02:18:02.049136876Z, version: 0, cause: TerminatedInstance }
19872024-08-28T02:18:02.672ZWARNSledAgent (InstanceManager): Halting and removing zone: oxz_propolis-server_8b43f99c-a0e7-445b-8ea1-1ee3b968f594 file = sled-agent/src/instance.rs:826 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594
19882024-08-28T02:18:04.408ZINFOSledAgent (InstanceManager): halt_and_remove_logged: Previous zone state: Running file = illumos-utils/src/zone.rs:270 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594
19892024-08-28T02:18:04.424ZINFOSledAgent (PortManager): Removed route set for subnet file = illumos-utils/src/opte/port_manager.rs:904 id = RouterId { vni: Vni(6178120), kind: System }
19902024-08-28T02:18:04.424ZINFOSledAgent (PortManager): Removed route set for subnet file = illumos-utils/src/opte/port_manager.rs:904 id = RouterId { vni: Vni(6178120), kind: Custom(V4(Ipv4Net { addr: 172.30.0.0, width: 22 })) }
19912024-08-28T02:18:04.425ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:514 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594 state = SledVmmState { vmm_state: VmmRuntimeState { state: Destroyed, gen: Generation(7), time_updated: 2024-08-28T02:18:04.425617066Z }, migration_in: None, migration_out: None }
19922024-08-28T02:18:04.477ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:514 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594 state = SledVmmState { vmm_state: VmmRuntimeState { state: Destroyed, gen: Generation(7), time_updated: 2024-08-28T02:18:04.425617066Z }, migration_in: None, migration_out: None }
19932024-08-28T02:18:04.481ZINFOSledAgent (InstanceManager): State monitoring task complete file = sled-agent/src/instance.rs:763 instance_id = 29444af2-b828-4c13-9cb7-17c7a56ed0e4 propolis_id = 8b43f99c-a0e7-445b-8ea1-1ee3b968f594
19942024-08-28T02:18:04.912ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:44940
19952024-08-28T02:18:04.912ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 42 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:44940 req_id = 71097645-4e1f-4ba4-8270-738d6716c48c response_code = 200 uri = /vpc-routes
19962024-08-28T02:18:04.919ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 95 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:44940 req_id = ad12e7e6-83f7-4509-9c01-e3d786bdbbab response_code = 204 uri = /vpc-routes
19972024-08-28T02:18:04.986ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:62896
19982024-08-28T02:18:04.986ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 136 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:62896 req_id = 81b1484d-69b6-4b9d-868d-f9a3bf3a3e5b response_code = 200 uri = /v2p
19992024-08-28T02:18:05.160ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:62895
20002024-08-28T02:18:05.160ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 156 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:62895 req_id = d3f06f0f-a3a0-4398-ae5a-c0b10e208fed response_code = 200 uri = /v2p
20012024-08-28T02:18:06.421ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:48926
20022024-08-28T02:18:06.422ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 638 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:48926 req_id = 608bbdc9-382a-46cc-aa97-31dc757a50be response_code = 200 uri = /vpc-routes
20032024-08-28T02:18:06.430ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 137 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:48926 req_id = 0d579555-ce45-4fdc-a17d-e294702eb6c6 response_code = 204 uri = /vpc-routes