Max level shown:
1 [ Nov 18 18:36:01 Enabled. ]
2 [ Nov 18 18:36:01 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/sled-agent/sled-agent run /opt/oxide/sled-agent/pkg/config.toml &"). ]
3 [ Nov 18 18:36:01 Method "start" exited with status 0. ]
4 [ Nov 18 18:36:01 Rereading configuration. ]
5 [ Nov 18 18:36:01 No 'refresh' method defined. Treating as :true. ]
6 note: configured to log to "/dev/stdout"
72024-11-18T18:36:01.660ZINFOSledAgent: Ensuring contract reaper thread file = illumos-utils/src/running_zone.rs:113
82024-11-18T18:36:01.660ZINFOSledAgent (BootstrapAgentStartup): finding links ["net0", "net1"] file = sled-agent/src/bootstrap/pre_server.rs:255
92024-11-18T18:36:01.712ZINFOSledAgent (BootstrapAgentStartup): Starting mg-ddm service addr-objs = [AddrObject { interface: "net0", name: "ll" }, AddrObject { interface: "net1", name: "ll" }] file = sled-agent/src/bootstrap/pre_server.rs:262
102024-11-18T18:36:01.712ZINFOSledAgent (BootstrapAgentStartup): Importing mg-ddm service file = sled-agent/src/bootstrap/maghemite.rs:42 path = /opt/oxide/mg-ddm/pkg/ddm/manifest.xml
112024-11-18T18:36:01.825ZINFOSledAgent (BootstrapAgentStartup): Setting mg-ddm interfaces file = sled-agent/src/bootstrap/maghemite.rs:50 interfaces = ("net0/ll" "net1/ll")
122024-11-18T18:36:01.834ZINFOSledAgent (BootstrapAgentStartup): Enabling mg-ddm service file = sled-agent/src/bootstrap/maghemite.rs:56
132024-11-18T18:36:01.842ZINFOSledAgent (BootstrapAgentStartup): Ensuring zfs key directory exists file = sled-agent/src/bootstrap/pre_server.rs:272 path = /var/run/oxide/
142024-11-18T18:36:02.448ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fdb0:18c0:4d0d:a02a::, width: 64 }
152024-11-18T18:36:02.585ZINFOSledAgent: Starting KeyManager file = sled-agent/src/long_running_tasks.rs:126
162024-11-18T18:36:02.586ZINFOSledAgent: Starting StorageManager file = sled-agent/src/long_running_tasks.rs:138
172024-11-18T18:36:02.586ZINFOSledAgent: Starting StorageMonitor file = sled-agent/src/long_running_tasks.rs:151
182024-11-18T18:36:02.586ZINFOSledAgent: Starting HardwareManager file = sled-agent/src/long_running_tasks.rs:173 nongimlet_observed_disks = [] sled_mode = Scrimlet { asic: SoftNpuZone }
192024-11-18T18:36:02.586ZINFOSledAgent (DumpSetup-worker): DumpSetup poll loop started. file = sled-agent/src/dump_setup.rs:520
202024-11-18T18:36:02.586ZINFOSledAgent (HardwareManager): Creating HardwareManager file = sled-hardware/src/illumos/mod.rs:717
212024-11-18T18:36:02.589ZINFOSledAgent (HardwareManager): Generated i86pc baseboard Pc { identifier: "sock", model: "i86pc" } file = sled-hardware/src/illumos/mod.rs:624
222024-11-18T18:36:02.589ZWARNSledAgent (HardwareManager): Device is not a Gimlet (i86pc), proceeding with null hardware view file = sled-hardware/src/illumos/mod.rs:766
232024-11-18T18:36:02.589ZINFOSledAgent: Starting HardwareMonitor file = sled-agent/src/long_running_tasks.rs:187
242024-11-18T18:36:02.589ZINFOSledAgent (HardwareManager): Monitoring for hardware updates file = sled-hardware/src/illumos/mod.rs:825
252024-11-18T18:36:02.589ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/m2_0.vdev
262024-11-18T18:36:02.589ZINFOSledAgent (HardwareMonitor): Checking current full hardware snapshot disks = {} file = sled-agent/src/hardware_monitor.rs:271 underlay_network_info = None
272024-11-18T18:36:02.589ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
282024-11-18T18:36:02.589ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
292024-11-18T18:36:02.589ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } path = /scratch/m2_0.vdev
302024-11-18T18:36:02.600ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/m2_0.vdev file = sled-hardware/src/disk.rs:367
312024-11-18T18:36:02.600ZINFOSledAgent (StorageResources): Formatting zpool with generated ID file = sled-hardware/src/disk.rs:378 id = 9216f0f9-29c9-4d33-8c71-0f237e1331a0 (zpool)
322024-11-18T18:36:02.736ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 9216f0f9-29c9-4d33-8c71-0f237e1331a0 (zpool), kind: Internal }
332024-11-18T18:36:03.305ZINFOSledAgent (StorageManager): Received DetectedRawDisksChanged { raw_disks: {}, tx: ... } file = sled-storage/src/manager.rs:603
342024-11-18T18:36:03.305ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/m2_1.vdev
352024-11-18T18:36:03.305ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
362024-11-18T18:36:03.305ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
372024-11-18T18:36:03.305ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
382024-11-18T18:36:03.305ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
392024-11-18T18:36:03.305ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" } path = /scratch/m2_1.vdev
402024-11-18T18:36:03.314ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/m2_1.vdev file = sled-hardware/src/disk.rs:367
412024-11-18T18:36:03.314ZINFOSledAgent (StorageResources): Formatting zpool with generated ID file = sled-hardware/src/disk.rs:378 id = c23ca4a6-dcd8-4835-a278-26664c492a4d (zpool)
422024-11-18T18:36:03.419ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: c23ca4a6-dcd8-4835-a278-26664c492a4d (zpool), kind: Internal }
432024-11-18T18:36:04.030ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_0.vdev
442024-11-18T18:36:04.030ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 9216f0f9-29c9-4d33-8c71-0f237e1331a0 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: c23ca4a6-dcd8-4835-a278-26664c492a4d (zpool), kind: Internal } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
452024-11-18T18:36:04.030ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
462024-11-18T18:36:04.030ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
472024-11-18T18:36:04.030ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
482024-11-18T18:36:04.030ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
492024-11-18T18:36:04.030ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_1.vdev
502024-11-18T18:36:04.030ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 9216f0f9-29c9-4d33-8c71-0f237e1331a0 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: c23ca4a6-dcd8-4835-a278-26664c492a4d (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
512024-11-18T18:36:04.030ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
522024-11-18T18:36:04.030ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
532024-11-18T18:36:04.030ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
542024-11-18T18:36:04.030ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
552024-11-18T18:36:04.030ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_2.vdev
562024-11-18T18:36:04.030ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
572024-11-18T18:36:04.030ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
582024-11-18T18:36:04.030ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 9216f0f9-29c9-4d33-8c71-0f237e1331a0 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: c23ca4a6-dcd8-4835-a278-26664c492a4d (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
592024-11-18T18:36:04.030ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
602024-11-18T18:36:04.030ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
612024-11-18T18:36:04.030ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_3.vdev
622024-11-18T18:36:04.030ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 9216f0f9-29c9-4d33-8c71-0f237e1331a0 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: c23ca4a6-dcd8-4835-a278-26664c492a4d (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
632024-11-18T18:36:04.030ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
642024-11-18T18:36:04.030ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
652024-11-18T18:36:04.030ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
662024-11-18T18:36:04.030ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
672024-11-18T18:36:04.030ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_4.vdev
682024-11-18T18:36:04.030ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 9216f0f9-29c9-4d33-8c71-0f237e1331a0 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: c23ca4a6-dcd8-4835-a278-26664c492a4d (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
692024-11-18T18:36:04.030ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
702024-11-18T18:36:04.030ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
712024-11-18T18:36:04.030ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
722024-11-18T18:36:04.030ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
732024-11-18T18:36:04.030ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_5.vdev
742024-11-18T18:36:04.030ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
752024-11-18T18:36:04.030ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
762024-11-18T18:36:04.030ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 9216f0f9-29c9-4d33-8c71-0f237e1331a0 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: c23ca4a6-dcd8-4835-a278-26664c492a4d (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
772024-11-18T18:36:04.030ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
782024-11-18T18:36:04.030ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
792024-11-18T18:36:04.030ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_6.vdev
802024-11-18T18:36:04.030ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
812024-11-18T18:36:04.030ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
822024-11-18T18:36:04.030ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 9216f0f9-29c9-4d33-8c71-0f237e1331a0 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: c23ca4a6-dcd8-4835-a278-26664c492a4d (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
832024-11-18T18:36:04.030ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_7.vdev
842024-11-18T18:36:04.030ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
852024-11-18T18:36:04.030ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
862024-11-18T18:36:04.030ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
872024-11-18T18:36:04.030ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
882024-11-18T18:36:04.030ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_8.vdev
892024-11-18T18:36:04.030ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
902024-11-18T18:36:04.030ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
912024-11-18T18:36:04.030ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 9216f0f9-29c9-4d33-8c71-0f237e1331a0 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: c23ca4a6-dcd8-4835-a278-26664c492a4d (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
922024-11-18T18:36:04.030ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
932024-11-18T18:36:04.030ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
942024-11-18T18:36:04.030ZINFOSledAgent: Waiting for boot disk file = sled-agent/src/long_running_tasks.rs:97
952024-11-18T18:36:04.030ZINFOSledAgent: Found boot disk DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } file = sled-agent/src/long_running_tasks.rs:99
962024-11-18T18:36:04.030ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 9216f0f9-29c9-4d33-8c71-0f237e1331a0 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: c23ca4a6-dcd8-4835-a278-26664c492a4d (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
972024-11-18T18:36:04.030ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
982024-11-18T18:36:04.030ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
992024-11-18T18:36:04.030ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
1002024-11-18T18:36:04.030ZINFOSledAgent: Starting Bootstore file = sled-agent/src/long_running_tasks.rs:211
1012024-11-18T18:36:04.030ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 9216f0f9-29c9-4d33-8c71-0f237e1331a0 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: c23ca4a6-dcd8-4835-a278-26664c492a4d (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
1022024-11-18T18:36:04.030ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
1032024-11-18T18:36:04.030ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
1042024-11-18T18:36:04.030ZINFOSledAgent (bootstore): No ledger in /pool/int/9216f0f9-29c9-4d33-8c71-0f237e1331a0/cluster/bootstore-fsm-state.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1052024-11-18T18:36:04.030ZINFOSledAgent (bootstore): No ledger in /pool/int/c23ca4a6-dcd8-4835-a278-26664c492a4d/cluster/bootstore-fsm-state.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1062024-11-18T18:36:04.030ZINFOSledAgent (bootstore): No ledger found. Loading Fsm::State as Uninitialized file = bootstore/src/schemes/v0/storage.rs:83 peer_id = pc-sock-i86pc
1072024-11-18T18:36:04.030ZINFOSledAgent (bootstore): No ledger in /pool/int/9216f0f9-29c9-4d33-8c71-0f237e1331a0/cluster/bootstore-network-config.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1082024-11-18T18:36:04.030ZINFOSledAgent (bootstore): No ledger in /pool/int/c23ca4a6-dcd8-4835-a278-26664c492a4d/cluster/bootstore-network-config.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1092024-11-18T18:36:04.030ZINFOSledAgent (bootstore): No ledger found for network config file = bootstore/src/schemes/v0/storage.rs:146 peer_id = pc-sock-i86pc
1102024-11-18T18:36:04.030ZINFOSledAgent: Starting Bootstore DDMD poller file = sled-agent/src/long_running_tasks.rs:216
1112024-11-18T18:36:04.030ZINFOSledAgent: Starting ZoneBundler related tasks file = sled-agent/src/long_running_tasks.rs:231
1122024-11-18T18:36:04.041ZINFOSledAgent (ServiceManager): Creating ServiceManager file = sled-agent/src/services.rs:736
1132024-11-18T18:36:04.042ZINFOSledAgent (auto-cleanup-task): top of bundle cleanup loop file = sled-agent/src/zone_bundle.rs:186 next_cleanup = Instant { tv_sec: 1154, tv_nsec: 372883727 } time_to_next_cleanup = 599.999935029s
1142024-11-18T18:36:04.042ZINFOSledAgent (ServiceManager): Ensuring scrimlet services (enabling services) file = sled-agent/src/services.rs:3948
1152024-11-18T18:36:04.042ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
1162024-11-18T18:36:04.044ZINFOSledAgent (BootstrapAgentStartup): No ledger in /pool/int/9216f0f9-29c9-4d33-8c71-0f237e1331a0/config/sled-agent-request.json file = common/src/ledger.rs:188
1172024-11-18T18:36:04.044ZINFOSledAgent (BootstrapAgentStartup): No ledger in /pool/int/c23ca4a6-dcd8-4835-a278-26664c492a4d/config/sled-agent-request.json file = common/src/ledger.rs:188
1182024-11-18T18:36:04.044ZINFOSledAgent (dropshot (BootstrapAgent)): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:327 local_addr = [fdb0:18c0:4d0d:a02a::1]:80
1192024-11-18T18:36:04.046ZINFOSledAgent (SledAgentSprocketsServer): Started listening file = sled-agent/src/bootstrap/sprockets_server.rs:51 local_addr = [fdb0:18c0:4d0d:a02a::1]:12346
1202024-11-18T18:36:04.047ZINFOSledAgent (RSS): Injecting RSS configuration: RackInitializeRequest { trust_quorum_peers: None, bootstrap_discovery: OnlyOurs, ntp_servers: [ "0.pool.ntp.org", ], dns_servers: [ 1.1.1.1, 9.9.9.9, ], internal_services_ip_pool_ranges: [ V4( Ipv4Range { first: 10.151.2.164, last: 10.151.2.173, }, ), ], external_dns_ips: [ 10.151.2.164, 10.151.2.165, ], external_dns_zone_name: "oxide.test", external_certificates: "<redacted>", recovery_silo: RecoverySiloConfig { silo_name: Name( "recovery", ), user_name: UserId( "recovery", ), user_password_hash: NewPasswordHash( PasswordHashString { string: "$argon2id$v=19$m=98304,t=13,p=1$RUlWc0ZxaHo0WFdrN0N6ZQ$S8p52j85GPvMhR/ek3GL0el/oProgTwWpHJZ8lsQQoY", encoding: B64, }, ), }, rack_network_config: RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [ PortConfigV2 { routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), nexthop: 10.151.2.100, vlan_id: None, rib_priority: None, }, ], addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.2.174, width: 24, }, ), vlan_id: None, }, ], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: Some( None, ), bgp_peers: [], autoneg: false, lldp: None, tx_eq: None, }, ], bgp: [], bfd: [], }, allowed_source_ips: Any, } file = sled-agent/src/rack_setup/service.rs:1200
1212024-11-18T18:36:04.047ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = internal-dns/resolver/src/resolver.rs:111
1222024-11-18T18:36:04.047ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
1232024-11-18T18:36:04.047ZINFOSledAgent (RSS): No ledger in /pool/int/9216f0f9-29c9-4d33-8c71-0f237e1331a0/config/rss-plan-completed.marker file = common/src/ledger.rs:188
1242024-11-18T18:36:04.047ZINFOSledAgent (RSS): No ledger in /pool/int/c23ca4a6-dcd8-4835-a278-26664c492a4d/config/rss-plan-completed.marker file = common/src/ledger.rs:188
1252024-11-18T18:36:04.047ZINFOSledAgent (RSS): RSS configuration has not been fully applied yet file = sled-agent/src/rack_setup/service.rs:1263
1262024-11-18T18:36:04.047ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
1272024-11-18T18:36:04.047ZINFOSledAgent (RSS): No ledger in /pool/int/9216f0f9-29c9-4d33-8c71-0f237e1331a0/config/rss-sled-plan.json file = common/src/ledger.rs:188
1282024-11-18T18:36:04.047ZINFOSledAgent (RSS): No ledger in /pool/int/c23ca4a6-dcd8-4835-a278-26664c492a4d/config/rss-sled-plan.json file = common/src/ledger.rs:188
1292024-11-18T18:36:04.047ZINFOSledAgent (RSS): Creating new allocation plan file = sled-agent/src/rack_setup/service.rs:1305
1302024-11-18T18:36:04.047ZINFOSledAgent (RSS): Serializing plan file = sled-agent/src/rack_setup/plan/sled.rs:156
1312024-11-18T18:36:04.047ZINFOSledAgent (RSS): Creating plan for the sled at fdb0:18c0:4d0d:a02a::1 file = sled-agent/src/rack_setup/plan/sled.rs:129
1322024-11-18T18:36:04.047ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
1332024-11-18T18:36:04.047ZINFOSledAgent (RSS): Writing ledger to /pool/int/9216f0f9-29c9-4d33-8c71-0f237e1331a0/config/.rss-sled-plan.json.tmp file = common/src/ledger.rs:199
1342024-11-18T18:36:04.047ZINFOSledAgent (RSS): Writing ledger to /pool/int/c23ca4a6-dcd8-4835-a278-26664c492a4d/config/.rss-sled-plan.json.tmp file = common/src/ledger.rs:199
1352024-11-18T18:36:04.047ZINFOSledAgent (RSS): Sled plan written to storage: Plan { rack_id: a5c736b2-440f-4c70-9b42-8f18e4718901, sleds: { [fdb0:18c0:4d0d:a02a::1]:12346: StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: ee2b3f44-b73f-4358-9df7-edb19f5079e2 (sled), rack_id: a5c736b2-440f-4c70-9b42-8f18e4718901, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64, }, }, }, }, }, config: RackInitializeRequest { trust_quorum_peers: None, bootstrap_discovery: OnlyOurs, ntp_servers: [ "0.pool.ntp.org", ], dns_servers: [ 1.1.1.1, 9.9.9.9, ], internal_services_ip_pool_ranges: [ V4( Ipv4Range { first: 10.151.2.164, last: 10.151.2.173, }, ), ], external_dns_ips: [ 10.151.2.164, 10.151.2.165, ], external_dns_zone_name: "oxide.test", external_certificates: "<redacted>", recovery_silo: RecoverySiloConfig { silo_name: Name( "recovery", ), user_name: UserId( "recovery", ), user_password_hash: NewPasswordHash( PasswordHashString { string: "$argon2id$v=19$m=98304,t=13,p=1$RUlWc0ZxaHo0WFdrN0N6ZQ$S8p52j85GPvMhR/ek3GL0el/oProgTwWpHJZ8lsQQoY", encoding: B64, }, ), }, rack_network_config: RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [ PortConfigV2 { routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), nexthop: 10.151.2.100, vlan_id: None, rib_priority: None, }, ], addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.2.174, width: 24, }, ), vlan_id: None, }, ], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: Some( None, ), bgp_peers: [], autoneg: false, lldp: None, tx_eq: None, }, ], bgp: [], bfd: [], }, allowed_source_ips: Any, }, } file = sled-agent/src/rack_setup/plan/sled.rs:176
1362024-11-18T18:36:04.047ZINFOSledAgent (RSS): Writing Rack Network Configuration to bootstore file = sled-agent/src/rack_setup/service.rs:1339
1372024-11-18T18:36:04.048ZINFOSledAgent (bootstore): Attempting to update network config with generation: 1, current_generation: 0 file = bootstore/src/schemes/v0/peer.rs:564 peer_id = pc-sock-i86pc
1382024-11-18T18:36:04.048ZINFOSledAgent (bootstore): Writing ledger to /pool/int/9216f0f9-29c9-4d33-8c71-0f237e1331a0/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
1392024-11-18T18:36:04.048ZINFOSledAgent (bootstore): Writing ledger to /pool/int/c23ca4a6-dcd8-4835-a278-26664c492a4d/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
1402024-11-18T18:36:04.048ZINFOSledAgent (bootstore): Broadcasting network config with generation 1 file = bootstore/src/schemes/v0/peer.rs:648 peer_id = pc-sock-i86pc
1412024-11-18T18:36:04.048ZINFOSledAgent (BootstrapAgentRssHandler): Received initialization request from RSS file = sled-agent/src/bootstrap/rss_handle.rs:232 request = StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: ee2b3f44-b73f-4358-9df7-edb19f5079e2 (sled), rack_id: a5c736b2-440f-4c70-9b42-8f18e4718901, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64 } } } } target_sled = [fdb0:18c0:4d0d:a02a::1]:12346
1422024-11-18T18:36:04.049ZINFOSledAgent (ServiceManager): Enabling switch zone (new) file = sled-agent/src/services.rs:4245
1432024-11-18T18:36:04.049ZINFOSledAgent (ServiceManager): Starting switch zone file = sled-agent/src/services.rs:4645
1442024-11-18T18:36:04.053ZINFOSledAgent (SledAgentSprocketsClient): Loaded keys and certs BootstrapAgentClient = [fdb0:18c0:4d0d:a02a::1]:12346 file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/client.rs:50
1452024-11-18T18:36:04.059ZINFOSledAgent (SledAgentSprocketsClient): Certificate chain verified successfully BootstrapAgentClient = [fdb0:18c0:4d0d:a02a::1]:12346 file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/keys.rs:307
1462024-11-18T18:36:04.059ZINFOSledAgent (SledAgentSprocketsClient): Signature verified successfully BootstrapAgentClient = [fdb0:18c0:4d0d:a02a::1]:12346 file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/keys.rs:376
1472024-11-18T18:36:04.063ZINFOSledAgent (SledAgentSprocketsServer): Certificate chain verified successfully file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/keys.rs:307
1482024-11-18T18:36:04.063ZINFOSledAgent (SledAgentSprocketsServer): Signature verified successfully file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/keys.rs:376
1492024-11-18T18:36:04.063ZINFOSledAgent (SledAgentSprocketsServer): Accepted connection file = sled-agent/src/bootstrap/sprockets_server.rs:74 remote_addr = [fdb0:18c0:4d0d:a02a::1]:37484
1502024-11-18T18:36:04.064ZINFOSledAgent (SledAgentMain): Loading Sled Agent: StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: ee2b3f44-b73f-4358-9df7-edb19f5079e2 (sled), rack_id: a5c736b2-440f-4c70-9b42-8f18e4718901, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64 } } } } file = sled-agent/src/bootstrap/server.rs:362
1512024-11-18T18:36:04.064ZINFOSledAgent (SledAgentMain): KeyManager: using hardcoded secret retriever file = sled-agent/src/bootstrap/server.rs:377
1522024-11-18T18:36:04.064ZINFOSledAgent: setting up sled agent server file = sled-agent/src/server.rs:44
1532024-11-18T18:36:04.064ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = internal-dns/resolver/src/resolver.rs:111
1542024-11-18T18:36:04.064ZINFOSledAgent (StorageManager): Received KeyManagerReady file = sled-storage/src/manager.rs:603
1552024-11-18T18:36:04.064ZINFOSledAgent (StorageManager): Loading storage config file = sled-storage/src/manager.rs:809
1562024-11-18T18:36:04.064ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:101::, width: 64 }
1572024-11-18T18:36:04.064ZINFOSledAgent (StorageManager): No ledger in /pool/int/9216f0f9-29c9-4d33-8c71-0f237e1331a0/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = load_disks_ledger
1582024-11-18T18:36:04.064ZINFOSledAgent (StorageManager): No ledger in /pool/int/c23ca4a6-dcd8-4835-a278-26664c492a4d/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = load_disks_ledger
1592024-11-18T18:36:04.064ZINFOSledAgent (StorageManager): No ledger of physical disks exists file = sled-storage/src/manager.rs:798
1602024-11-18T18:36:04.064ZINFOSledAgent (StorageManager): KeyManager ready, but no ledger detected file = sled-storage/src/manager.rs:826
1612024-11-18T18:36:04.064ZINFOSledAgent (StorageManager): automatically managing disks file = sled-storage/src/manager.rs:592
1622024-11-18T18:36:04.064ZINFOSledAgent (StorageResources): Synchronizing disk managment file = sled-storage/src/resources.rs:302
1632024-11-18T18:36:04.064ZINFOSledAgent (StorageManager): Successfully synchronized disks without error file = sled-storage/src/manager.rs:697 result = DisksManagementResult { status: [] }
1642024-11-18T18:36:04.080ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
1652024-11-18T18:36:04.100ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_switch file = illumos-utils/src/zone.rs:325
1662024-11-18T18:36:04.116ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_switch file = illumos-utils/src/zone.rs:363
1672024-11-18T18:36:04.123ZINFOSledAgent: SledAgent::new(..) starting file = sled-agent/src/sled_agent.rs:408 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
1682024-11-18T18:36:04.124ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
1692024-11-18T18:36:04.124ZINFOSledAgent: Requested swap device of size 64 GiB file = sled-agent/src/sled_agent.rs:420 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
1702024-11-18T18:36:04.124ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1712024-11-18T18:36:04.126ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 107.006953ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "98a48258-46cd-4325-9ef6-a9e7d5a96a8a", "content-length": "124", "date": "Mon, 18 Nov 2024 18:36:04 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "98a48258-46cd-4325-9ef6-a9e7d5a96a8a" } file = clients/ddm-admin-client/src/lib.rs:177
1722024-11-18T18:36:04.137ZINFOSledAgent: attempting to create encrypted zvol: name="oxi_9216f0f9-29c9-4d33-8c71-0f237e1331a0/swap", size_gb=64 file = sled-agent/src/swap_device.rs:153
1732024-11-18T18:36:04.169ZINFOSledAgent: successfully created encrypted zvol: name="oxi_9216f0f9-29c9-4d33-8c71-0f237e1331a0/swap", size_gb=64 file = sled-agent/src/swap_device.rs:249
1742024-11-18T18:36:04.170ZINFOSledAgent: adding swap device: swapname="/dev/zvol/dsk/oxi_9216f0f9-29c9-4d33-8c71-0f237e1331a0/swap" file = sled-agent/src/swap_device.rs:96
1752024-11-18T18:36:04.175ZINFOSledAgent: Mounting backing filesystems file = sled-agent/src/sled_agent.rs:435 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
1762024-11-18T18:36:04.175ZINFOSledAgent (BackingFs): Processing fmd file = sled-agent/src/backing_fs.rs:128
1772024-11-18T18:36:04.175ZINFOSledAgent (BackingFs): Ensuring dataset oxi_9216f0f9-29c9-4d33-8c71-0f237e1331a0/backing/fmd file = sled-agent/src/backing_fs.rs:138
1782024-11-18T18:36:04.235ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1792024-11-18T18:36:04.235ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 150.857103ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "a1f31aa4-e4bc-451c-9129-68b3f0bfd13c", "content-length": "124", "date": "Mon, 18 Nov 2024 18:36:04 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "a1f31aa4-e4bc-451c-9129-68b3f0bfd13c" } file = clients/ddm-admin-client/src/lib.rs:177
1802024-11-18T18:36:04.283ZINFOSledAgent (BackingFs): Stopping service svc:/system/fmd:default file = sled-agent/src/backing_fs.rs:173
1812024-11-18T18:36:04.388ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1822024-11-18T18:36:04.389ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 192.504586ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "0d540326-0dc2-471d-89a9-8ffb0d3ed4d0", "content-length": "124", "date": "Mon, 18 Nov 2024 18:36:04 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "0d540326-0dc2-471d-89a9-8ffb0d3ed4d0" } file = clients/ddm-admin-client/src/lib.rs:177
1832024-11-18T18:36:04.401ZINFOSledAgent (BackingFs): Mounting oxi_9216f0f9-29c9-4d33-8c71-0f237e1331a0/backing/fmd on /var/fm/fmd file = sled-agent/src/backing_fs.rs:181
1842024-11-18T18:36:04.415ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/rsrc file = sled-agent/src/backing_fs.rs:189
1852024-11-18T18:36:04.415ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/ckpt file = sled-agent/src/backing_fs.rs:189
1862024-11-18T18:36:04.415ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/xprt file = sled-agent/src/backing_fs.rs:189
1872024-11-18T18:36:04.416ZINFOSledAgent (BackingFs): Starting service svc:/system/fmd:default file = sled-agent/src/backing_fs.rs:195
1882024-11-18T18:36:04.583ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1892024-11-18T18:36:04.584ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 254.644819ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "bdace084-de34-487c-b950-b7c88cb6473c", "content-length": "124", "date": "Mon, 18 Nov 2024 18:36:04 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "bdace084-de34-487c-b950-b7c88cb6473c" } file = clients/ddm-admin-client/src/lib.rs:177
1902024-11-18T18:36:04.839ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1912024-11-18T18:36:04.840ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 165.590861ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "019cf7cc-6f43-4975-b9ac-1d6efb1fd688", "content-length": "124", "date": "Mon, 18 Nov 2024 18:36:04 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "019cf7cc-6f43-4975-b9ac-1d6efb1fd688" } file = clients/ddm-admin-client/src/lib.rs:177
1922024-11-18T18:36:05.006ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1932024-11-18T18:36:05.007ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 159.582933ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "c2d0b3bb-af19-4fb8-bdbf-cefccef4633f", "content-length": "124", "date": "Mon, 18 Nov 2024 18:36:04 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "c2d0b3bb-af19-4fb8-bdbf-cefccef4633f" } file = clients/ddm-admin-client/src/lib.rs:177
1942024-11-18T18:36:05.149ZINFOSledAgent: using '[AddrObject { interface: "net0", name: "ll" }, AddrObject { interface: "net1", name: "ll" }]' as data links for xde driver file = illumos-utils/src/opte/illumos.rs:93 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
1952024-11-18T18:36:05.155ZINFOSledAgent (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:327 local_addr = [fd00:1122:3344:101::1]:48709 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
1962024-11-18T18:36:05.155ZINFOSledAgent (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = internal-dns/resolver/src/resolver.rs:111 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
1972024-11-18T18:36:05.155ZINFOSledAgent (producer-server): starting oximeter metric producer server address = [fd00:1122:3344:101::1]:48709 file = oximeter/producer/src/lib.rs:279 interval = 30s producer_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
1982024-11-18T18:36:05.156ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 143.807964ms error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
1992024-11-18T18:36:05.157ZINFOSledAgent (VmmReservoirManager): 50% of 102952026112 physical ram = 51476013056 bytes) file = sled-agent/src/vmm_reservoir.rs:267 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
2002024-11-18T18:36:05.157ZINFOSledAgent (VmmReservoirManager): Setting reservoir size to 49090 MiB bytes file = sled-agent/src/vmm_reservoir.rs:275 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
2012024-11-18T18:36:05.157ZINFOSledAgent (ServiceManager): sled agent started file = sled-agent/src/services.rs:920 underlay_address = fd00:1122:3344:101::1
2022024-11-18T18:36:05.157ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = internal-dns/resolver/src/resolver.rs:111
2032024-11-18T18:36:05.167ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
2042024-11-18T18:36:05.301ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 545.706687ms error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
2052024-11-18T18:36:05.848ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 529.555576ms error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
2062024-11-18T18:36:06.379ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 1.534493933s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
2072024-11-18T18:36:07.686ZINFOSledAgent (ServiceManager): Setting up dendrite service file = sled-agent/src/services.rs:2669
2082024-11-18T18:36:07.686ZINFOSledAgent (ServiceManager): Setting up lldpd service file = sled-agent/src/services.rs:2914
2092024-11-18T18:36:07.686ZINFOSledAgent (ServiceManager): Setting up MGS service file = sled-agent/src/services.rs:2539
2102024-11-18T18:36:07.686ZINFOSledAgent (ServiceManager): Setting up wicketd service file = sled-agent/src/services.rs:2586
2112024-11-18T18:36:07.686ZINFOSledAgent (ServiceManager): Setting up mgd service file = sled-agent/src/services.rs:2995
2122024-11-18T18:36:07.686ZINFOSledAgent (ServiceManager): Setting up mg-ddm service file = sled-agent/src/services.rs:3040
2132024-11-18T18:36:07.688ZINFOSledAgent (ServiceManager): Setting up tfport service file = sled-agent/src/services.rs:2839
2142024-11-18T18:36:07.690ZINFOSledAgent (ServiceManager): Setting up Simulated SP service file = sled-agent/src/services.rs:2580
2152024-11-18T18:36:07.690ZINFOSledAgent (ServiceManager): Profile for oxz_switch: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService0'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/mgs"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="address" value='[::1]:12225'/> <propval type="astring" name="id" value='a77ae76f-e7f1-4199-bba7-84aeb497f74b'/> <propval type="astring" name="rack_id" value='a5c736b2-440f-4c70-9b42-8f18e4718901'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/wicketd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="address" value='[::1]:12226'/> <propval type="astring" name="artifact-address" value='[fdb0:18c0:4d0d:a02a::2]:12227'/> <propval type="astring" name="baseboard-file" value='/opt/oxide/baseboard.json'/> <propval type="astring" name="mgs-address" value='[::1]:12225'/> <propval type="astring" name="nexus-proxy-address" value='[::]:12229'/> <propval type="astring" name="rack-subnet" value='fd00:1122:3344::'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/switch_zone_setup"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="gz_local_link_addr" value='fe80::8:20ff:febe:1eda'/> <property type="astring" name="link_local_links"> <astring_list> <value_node value="tfportrear0_0" /> <value_node value="oxBootstrap0" /> </astring_list> </property> <propval type="astring" name="bootstrap_addr" value='fdb0:18c0:4d0d:a02a::2'/> <propval type="astring" name="bootstrap_vnic" value='oxBootstrap0'/> <propval type="astring" name="baseboard_info" value='{"type":"pc","identifier":"sock","model":"i86pc"}'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/dendrite"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="rack_id" value='a5c736b2-440f-4c70-9b42-8f18e4718901'/> <propval type="astring" name="sled_id" value='ee2b3f44-b73f-4358-9df7-edb19f5079e2'/> <propval type="astring" name="sled_model" value='i86pc'/> <propval type="astring" name="sled_serial" value='sock'/> <propval type="astring" name="sled_revision" value='0'/> <propval type="astring" name="address" value='[::1]:12224'/> <propval type="astring" name="front_ports" value='1'/> <propval type="astring" name="rear_ports" value='1'/> <propval type="astring" name="port_config" value='/opt/oxide/dendrite/misc/softnpu_single_sled_config.toml'/> <propval type="astring" name="mgmt" value='uds'/> <propval type="astring" name="uds_path" value='/opt/softnpu/stuff'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/tfport"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="host" value='[::1]'/> <propval type="astring" name="port" value='12224'/> <propval type="astring" name="flags" value='--sync-only'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/lldpd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="board_rev" value='softnpu_front_1_rear_1'/> <propval type="astring" name="scrimlet_id" value='sock'/> <propval type="astring" name="scrimlet_model" value='i86pc'/> <propval type="astring" name="address" value='[::1]:12230'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/pumpkind"> </service> <service version="1" type="service" name="oxide/mgd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="sled_uuid" value='ee2b3f44-b73f-4358-9df7-edb19f5079e2'/> <propval type="astring" name="rack_uuid" value='a5c736b2-440f-4c70-9b42-8f18e4718901'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/mg-ddm"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="mode" value='transit'/> <propval type="astring" name="dendrite" value='true'/> <propval type="astring" name="sled_uuid" value='ee2b3f44-b73f-4358-9df7-edb19f5079e2'/> <propval type="astring" name="rack_uuid" value='a5c736b2-440f-4c70-9b42-8f18e4718901'/> <propval type="astring" name="interfaces" value='tfportrear0_0/ll'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/uplink"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
2162024-11-18T18:36:07.690ZINFOSledAgent (ServiceManager): Booting oxz_switch zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_switch
2172024-11-18T18:36:07.949ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 38.861377ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2182024-11-18T18:36:07.951ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 5.364019063s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
2192024-11-18T18:36:08.132ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 83.342075ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2202024-11-18T18:36:08.311ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 294.392431ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2212024-11-18T18:36:08.665ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 498.520086ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2222024-11-18T18:36:09.271ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 953.147527ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2232024-11-18T18:36:10.428ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 874.117571ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2242024-11-18T18:36:11.537ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 868.417187ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2252024-11-18T18:36:12.504ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 764.966925ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2262024-11-18T18:36:13.734ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 10.834992941s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
2272024-11-18T18:36:13.750ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/ee2b3f44-b73f-4358-9df7-edb19f5079e2) file = sled-agent/src/nexus.rs:527 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
2282024-11-18T18:36:13.780ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2292024-11-18T18:36:13.780ZINFOSledAgent (dropshot (Repo Depot)): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:327 local_addr = [fd00:1122:3344:101::1]:12348 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
2302024-11-18T18:36:13.782ZINFOSledAgent (dropshot (SledAgent)): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:327 local_addr = [fd00:1122:3344:101::1]:12345
2312024-11-18T18:36:13.782ZINFOSledAgent (SledAgentMain): Sled Agent loaded; recording configuration file = sled-agent/src/bootstrap/server.rs:432
2322024-11-18T18:36:13.782ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2332024-11-18T18:36:13.782ZINFOSledAgent (SledAgentMain): Writing ledger to /pool/int/9216f0f9-29c9-4d33-8c71-0f237e1331a0/config/.sled-agent-request.json.tmp file = common/src/ledger.rs:199
2342024-11-18T18:36:13.782ZINFOSledAgent (SledAgentMain): Writing ledger to /pool/int/c23ca4a6-dcd8-4835-a278-26664c492a4d/config/.sled-agent-request.json.tmp file = common/src/ledger.rs:199
2352024-11-18T18:36:13.782ZINFOSledAgent (HardwareMonitor): Sled Agent Started file = sled-agent/src/hardware_monitor.rs:138
2362024-11-18T18:36:13.782ZINFOSledAgent (HardwareMonitor): Checking current full hardware snapshot disks = {} file = sled-agent/src/hardware_monitor.rs:271 underlay_network_info = Some((fd00:1122:3344:101::2, Some(RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56 }, infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [PortConfigV2 { routes: [RouteConfig { destination: V4(Ipv4Net { addr: 0.0.0.0, width: 0 }), nexthop: 10.151.2.100, vlan_id: None, rib_priority: None }], addresses: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: Some(None), bgp_peers: [], autoneg: false, lldp: None, tx_eq: None }], bgp: [], bfd: [] })))
2372024-11-18T18:36:13.782ZINFOSledAgent (ServiceManager): Ensuring scrimlet services (enabling services) file = sled-agent/src/services.rs:3948
2382024-11-18T18:36:13.782ZINFOSledAgent (SledAgentSprocketsServer): Connection closed file = sled-agent/src/bootstrap/sprockets_server.rs:81 remote_addr = [fdb0:18c0:4d0d:a02a::1]:37484
2392024-11-18T18:36:13.986ZINFOSledAgent (BootstrapAgentRssHandler): Peer agent initialized file = sled-agent/src/bootstrap/rss_handle.rs:117 peer = [fdb0:18c0:4d0d:a02a::1]:12346
2402024-11-18T18:36:13.986ZINFOSledAgent (BootstrapAgentRssHandler): Initialized sled agent file = sled-agent/src/bootstrap/rss_handle.rs:252 target_sled = [fdb0:18c0:4d0d:a02a::1]:12346
2412024-11-18T18:36:13.986ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2422024-11-18T18:36:13.986ZINFOSledAgent (RSS): No ledger in /pool/int/9216f0f9-29c9-4d33-8c71-0f237e1331a0/config/rss-service-plan-v5.json file = common/src/ledger.rs:188
2432024-11-18T18:36:13.986ZINFOSledAgent (RSS): No ledger in /pool/int/c23ca4a6-dcd8-4835-a278-26664c492a4d/config/rss-service-plan-v5.json file = common/src/ledger.rs:188
2442024-11-18T18:36:13.986ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2452024-11-18T18:36:13.986ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2462024-11-18T18:36:13.986ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2472024-11-18T18:36:13.986ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2482024-11-18T18:36:13.991ZINFOSledAgent (ServiceManager): Re-enabling running switch zone (new address) file = sled-agent/src/services.rs:4269 new = [fd00:1122:3344:101::2, ::1] old = [::1]
2492024-11-18T18:36:13.991ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::2 exists file = sled-agent/src/services.rs:4285
2502024-11-18T18:36:13.992ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:101::2, prefix: 64 })) file = illumos-utils/src/running_zone.rs:534 zone = oxz_switch
2512024-11-18T18:36:14.288ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:39282
2522024-11-18T18:36:14.288ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2532024-11-18T18:36:14.288ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2542024-11-18T18:36:14.288ZINFOSledAgent (ServiceManager): No ledger in /pool/int/9216f0f9-29c9-4d33-8c71-0f237e1331a0/config/omicron-zones.json file = common/src/ledger.rs:188
2552024-11-18T18:36:14.288ZINFOSledAgent (ServiceManager): No ledger in /pool/int/c23ca4a6-dcd8-4835-a278-26664c492a4d/config/omicron-zones.json file = common/src/ledger.rs:188
2562024-11-18T18:36:14.288ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 314 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:39282 req_id = a434dd38-edb7-40dd-9232-4cf0b35744e7 response_code = 200 uri = /inventory
2572024-11-18T18:36:14.318ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:43192
2582024-11-18T18:36:14.318ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 56 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:43192 req_id = 16104e5a-9e59-42ad-94b4-89e80065d47c response_code = 200 uri = /sled-role
2592024-11-18T18:36:14.319ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2602024-11-18T18:36:14.319ZINFOSledAgent (RSS): Writing ledger to /pool/int/9216f0f9-29c9-4d33-8c71-0f237e1331a0/config/.rss-service-plan-v5.json.tmp file = common/src/ledger.rs:199
2612024-11-18T18:36:14.320ZINFOSledAgent (RSS): Writing ledger to /pool/int/c23ca4a6-dcd8-4835-a278-26664c492a4d/config/.rss-service-plan-v5.json.tmp file = common/src/ledger.rs:199
2622024-11-18T18:36:14.320ZINFOSledAgent (RSS): Service plan written to storage file = sled-agent/src/rack_setup/plan/service.rs:980
2632024-11-18T18:36:14.353ZINFOSledAgent (RSS): attempting to set up sled's storage: OmicronPhysicalDisksConfig { generation: Generation(1), disks: [OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, id: 58bded64-08c1-4d43-812e-7e07f243071c, pool_id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, id: 158e972a-8497-44c7-bb1e-01ef3fce3ba9, pool_id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, id: 62411948-c171-4e02-a1f1-68a69b50e591, pool_id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, id: fd914410-cd57-4e75-a3bb-5ab2261d9ace, pool_id: 0199d4fd-4090-4a7c-83e6-040814eb79bb (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, id: 476660d2-4cb9-4920-a658-8150a22c77fa, pool_id: 5840d858-3782-45e1-a6e0-d60d070ad2c3 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, id: a8e26a7a-46f8-4245-a6a7-2c20d7768174, pool_id: 131b4d69-dbb0-4397-9284-176e4a3c973d (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, id: 4eb4c37d-2da9-4a41-ac70-f8ea922d0c30, pool_id: 453f7245-5f70-4fe8-b37e-054e56d5c1c5 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, id: f63f7290-924c-4929-915e-4b943eec2124, pool_id: b09b6e80-5341-4ded-a520-3f312403e198 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, id: a94fc479-9b00-46f7-a845-7e34073ae53e, pool_id: f23dc95a-d65a-4537-b65e-161954e059fa (zpool) }] } file = sled-agent/src/rack_setup/service.rs:382 sled_address = [fd00:1122:3344:101::1]:12345
2642024-11-18T18:36:14.353ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:62500
2652024-11-18T18:36:14.353ZINFOSledAgent: physical disks ensure file = sled-agent/src/sled_agent.rs:884 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
2662024-11-18T18:36:14.353ZINFOSledAgent (StorageManager): Received OmicronPhysicalDisksEnsure { config: OmicronPhysicalDisksConfig { generation: Generation(1), disks: [OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, id: 58bded64-08c1-4d43-812e-7e07f243071c, pool_id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, id: 158e972a-8497-44c7-bb1e-01ef3fce3ba9, pool_id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, id: 62411948-c171-4e02-a1f1-68a69b50e591, pool_id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, id: fd914410-cd57-4e75-a3bb-5ab2261d9ace, pool_id: 0199d4fd-4090-4a7c-83e6-040814eb79bb (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, id: 476660d2-4cb9-4920-a658-8150a22c77fa, pool_id: 5840d858-3782-45e1-a6e0-d60d070ad2c3 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, id: a8e26a7a-46f8-4245-a6a7-2c20d7768174, pool_id: 131b4d69-dbb0-4397-9284-176e4a3c973d (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, id: 4eb4c37d-2da9-4a41-ac70-f8ea922d0c30, pool_id: 453f7245-5f70-4fe8-b37e-054e56d5c1c5 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, id: f63f7290-924c-4929-915e-4b943eec2124, pool_id: b09b6e80-5341-4ded-a520-3f312403e198 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, id: a94fc479-9b00-46f7-a845-7e34073ae53e, pool_id: f23dc95a-d65a-4537-b65e-161954e059fa (zpool) }] }, tx: ... } file = sled-storage/src/manager.rs:603
2672024-11-18T18:36:14.353ZINFOSledAgent (StorageManager): No ledger in /pool/int/9216f0f9-29c9-4d33-8c71-0f237e1331a0/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = omicron_physical_disks_ensure
2682024-11-18T18:36:14.353ZINFOSledAgent (StorageManager): No ledger in /pool/int/c23ca4a6-dcd8-4835-a278-26664c492a4d/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = omicron_physical_disks_ensure
2692024-11-18T18:36:14.353ZINFOSledAgent (StorageManager): No previously-stored 'requested disks', creating new ledger file = sled-storage/src/manager.rs:1171 request = omicron_physical_disks_ensure
2702024-11-18T18:36:14.353ZINFOSledAgent (StorageResources): Synchronizing disk managment file = sled-storage/src/resources.rs:302
2712024-11-18T18:36:14.353ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:342
2722024-11-18T18:36:14.353ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
2732024-11-18T18:36:14.353ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } path = /scratch/u2_0.vdev
2742024-11-18T18:36:14.490ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_0.vdev file = sled-hardware/src/disk.rs:367
2752024-11-18T18:36:14.490ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 4be98a0c-ab4c-4309-948b-04d776603876 (zpool)
2762024-11-18T18:36:14.943ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/ee2b3f44-b73f-4358-9df7-edb19f5079e2) file = sled-agent/src/probe_manager.rs:231 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
2772024-11-18T18:36:15.164ZINFOSledAgent (VmmReservoirManager): Finished setting reservoir size to 49090 MiB bytes file = sled-agent/src/vmm_reservoir.rs:279 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
2782024-11-18T18:36:15.165ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 12802533969288132606 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
2792024-11-18T18:36:15.165ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 9234514238289780840 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
2802024-11-18T18:36:15.216ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }
2812024-11-18T18:36:15.225ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:240
2822024-11-18T18:36:15.225ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:242
2832024-11-18T18:36:15.225ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:246
2842024-11-18T18:36:15.225ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:248
2852024-11-18T18:36:15.225ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key")
2862024-11-18T18:36:15.225ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key")
2872024-11-18T18:36:15.225ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
2882024-11-18T18:36:15.269ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
2892024-11-18T18:36:15.276ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_4be98a0c-ab4c-4309-948b-04d776603876/crypt/zone file = sled-storage/src/dataset.rs:303
2902024-11-18T18:36:15.489ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_4be98a0c-ab4c-4309-948b-04d776603876 file = sled-storage/src/dataset.rs:389
2912024-11-18T18:36:15.497ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:440
2922024-11-18T18:36:15.497ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:357
2932024-11-18T18:36:15.497ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:342
2942024-11-18T18:36:15.497ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
2952024-11-18T18:36:15.497ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } path = /scratch/u2_1.vdev
2962024-11-18T18:36:15.507ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_1.vdev file = sled-hardware/src/disk.rs:367
2972024-11-18T18:36:15.507ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool)
2982024-11-18T18:36:15.630ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }
2992024-11-18T18:36:15.638ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:240
3002024-11-18T18:36:15.638ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:242
3012024-11-18T18:36:15.638ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:246
3022024-11-18T18:36:15.638ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:248
3032024-11-18T18:36:15.639ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key")
3042024-11-18T18:36:15.639ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key")
3052024-11-18T18:36:15.639ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
3062024-11-18T18:36:15.680ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3072024-11-18T18:36:15.688ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_4f35fd7e-936a-4e09-946b-7cb174580897/crypt/zone file = sled-storage/src/dataset.rs:303
3082024-11-18T18:36:15.908ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_4f35fd7e-936a-4e09-946b-7cb174580897 file = sled-storage/src/dataset.rs:389
3092024-11-18T18:36:15.916ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:440
3102024-11-18T18:36:15.916ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:357
3112024-11-18T18:36:15.916ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:342
3122024-11-18T18:36:15.916ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3132024-11-18T18:36:15.916ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } path = /scratch/u2_2.vdev
3142024-11-18T18:36:15.926ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_2.vdev file = sled-hardware/src/disk.rs:367
3152024-11-18T18:36:15.926ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = df4faf1b-f747-4cda-8481-deaa8523a533 (zpool)
3162024-11-18T18:36:16.044ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }
3172024-11-18T18:36:16.052ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/dataset.rs:240
3182024-11-18T18:36:16.052ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:242
3192024-11-18T18:36:16.052ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:246
3202024-11-18T18:36:16.052ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:248
3212024-11-18T18:36:16.052ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key")
3222024-11-18T18:36:16.052ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key")
3232024-11-18T18:36:16.052ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
3242024-11-18T18:36:16.056ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/ee2b3f44-b73f-4358-9df7-edb19f5079e2) file = sled-agent/src/probe_manager.rs:231 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
3252024-11-18T18:36:16.096ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3262024-11-18T18:36:16.103ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_df4faf1b-f747-4cda-8481-deaa8523a533/crypt/zone file = sled-storage/src/dataset.rs:303
3272024-11-18T18:36:16.318ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_df4faf1b-f747-4cda-8481-deaa8523a533 file = sled-storage/src/dataset.rs:389
3282024-11-18T18:36:16.326ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:440
3292024-11-18T18:36:16.326ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:357
3302024-11-18T18:36:16.326ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:342
3312024-11-18T18:36:16.326ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3322024-11-18T18:36:16.326ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } path = /scratch/u2_3.vdev
3332024-11-18T18:36:16.336ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_3.vdev file = sled-hardware/src/disk.rs:367
3342024-11-18T18:36:16.336ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 0199d4fd-4090-4a7c-83e6-040814eb79bb (zpool)
3352024-11-18T18:36:16.457ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 0199d4fd-4090-4a7c-83e6-040814eb79bb (zpool), kind: External }
3362024-11-18T18:36:16.465ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/dataset.rs:240
3372024-11-18T18:36:16.465ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:242
3382024-11-18T18:36:16.465ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:246
3392024-11-18T18:36:16.465ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:248
3402024-11-18T18:36:16.465ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key")
3412024-11-18T18:36:16.465ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key")
3422024-11-18T18:36:16.465ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
3432024-11-18T18:36:16.509ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3442024-11-18T18:36:16.516ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_0199d4fd-4090-4a7c-83e6-040814eb79bb/crypt/zone file = sled-storage/src/dataset.rs:303
3452024-11-18T18:36:16.735ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_0199d4fd-4090-4a7c-83e6-040814eb79bb file = sled-storage/src/dataset.rs:389
3462024-11-18T18:36:16.743ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:440
3472024-11-18T18:36:16.743ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:357
3482024-11-18T18:36:16.743ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:342
3492024-11-18T18:36:16.743ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3502024-11-18T18:36:16.743ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } path = /scratch/u2_4.vdev
3512024-11-18T18:36:16.754ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_4.vdev file = sled-hardware/src/disk.rs:367
3522024-11-18T18:36:16.754ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 5840d858-3782-45e1-a6e0-d60d070ad2c3 (zpool)
3532024-11-18T18:36:16.869ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 5840d858-3782-45e1-a6e0-d60d070ad2c3 (zpool), kind: External }
3542024-11-18T18:36:16.877ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/dataset.rs:240
3552024-11-18T18:36:16.877ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:242
3562024-11-18T18:36:16.877ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:246
3572024-11-18T18:36:16.877ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:248
3582024-11-18T18:36:16.877ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key")
3592024-11-18T18:36:16.877ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key")
3602024-11-18T18:36:16.877ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
3612024-11-18T18:36:16.918ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3622024-11-18T18:36:16.926ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_5840d858-3782-45e1-a6e0-d60d070ad2c3/crypt/zone file = sled-storage/src/dataset.rs:303
3632024-11-18T18:36:17.059ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/ee2b3f44-b73f-4358-9df7-edb19f5079e2) file = sled-agent/src/probe_manager.rs:231 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
3642024-11-18T18:36:17.144ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_5840d858-3782-45e1-a6e0-d60d070ad2c3 file = sled-storage/src/dataset.rs:389
3652024-11-18T18:36:17.152ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:440
3662024-11-18T18:36:17.152ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:357
3672024-11-18T18:36:17.152ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:342
3682024-11-18T18:36:17.152ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3692024-11-18T18:36:17.152ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } path = /scratch/u2_5.vdev
3702024-11-18T18:36:17.163ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_5.vdev file = sled-hardware/src/disk.rs:367
3712024-11-18T18:36:17.163ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 131b4d69-dbb0-4397-9284-176e4a3c973d (zpool)
3722024-11-18T18:36:17.292ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 131b4d69-dbb0-4397-9284-176e4a3c973d (zpool), kind: External }
3732024-11-18T18:36:17.300ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/dataset.rs:240
3742024-11-18T18:36:17.300ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:242
3752024-11-18T18:36:17.300ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:246
3762024-11-18T18:36:17.300ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:248
3772024-11-18T18:36:17.300ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key")
3782024-11-18T18:36:17.300ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key")
3792024-11-18T18:36:17.300ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
3802024-11-18T18:36:17.345ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3812024-11-18T18:36:17.353ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_131b4d69-dbb0-4397-9284-176e4a3c973d/crypt/zone file = sled-storage/src/dataset.rs:303
3822024-11-18T18:36:17.581ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_131b4d69-dbb0-4397-9284-176e4a3c973d file = sled-storage/src/dataset.rs:389
3832024-11-18T18:36:17.590ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:440
3842024-11-18T18:36:17.590ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:357
3852024-11-18T18:36:17.590ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:342
3862024-11-18T18:36:17.590ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3872024-11-18T18:36:17.590ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } path = /scratch/u2_6.vdev
3882024-11-18T18:36:17.599ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_6.vdev file = sled-hardware/src/disk.rs:367
3892024-11-18T18:36:17.599ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 453f7245-5f70-4fe8-b37e-054e56d5c1c5 (zpool)
3902024-11-18T18:36:17.734ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 453f7245-5f70-4fe8-b37e-054e56d5c1c5 (zpool), kind: External }
3912024-11-18T18:36:17.741ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/dataset.rs:240
3922024-11-18T18:36:17.741ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:242
3932024-11-18T18:36:17.741ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:246
3942024-11-18T18:36:17.741ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:248
3952024-11-18T18:36:17.742ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key")
3962024-11-18T18:36:17.742ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key")
3972024-11-18T18:36:17.742ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
3982024-11-18T18:36:17.785ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3992024-11-18T18:36:17.793ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_453f7245-5f70-4fe8-b37e-054e56d5c1c5/crypt/zone file = sled-storage/src/dataset.rs:303
4002024-11-18T18:36:17.997ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_453f7245-5f70-4fe8-b37e-054e56d5c1c5 file = sled-storage/src/dataset.rs:389
4012024-11-18T18:36:18.005ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:440
4022024-11-18T18:36:18.005ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:357
4032024-11-18T18:36:18.005ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:342
4042024-11-18T18:36:18.005ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
4052024-11-18T18:36:18.005ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } path = /scratch/u2_7.vdev
4062024-11-18T18:36:18.016ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_7.vdev file = sled-hardware/src/disk.rs:367
4072024-11-18T18:36:18.016ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = b09b6e80-5341-4ded-a520-3f312403e198 (zpool)
4082024-11-18T18:36:18.136ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: b09b6e80-5341-4ded-a520-3f312403e198 (zpool), kind: External }
4092024-11-18T18:36:18.145ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/dataset.rs:240
4102024-11-18T18:36:18.145ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:242
4112024-11-18T18:36:18.145ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:246
4122024-11-18T18:36:18.145ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:248
4132024-11-18T18:36:18.145ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key")
4142024-11-18T18:36:18.145ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key")
4152024-11-18T18:36:18.145ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
4162024-11-18T18:36:18.149ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/ee2b3f44-b73f-4358-9df7-edb19f5079e2) file = sled-agent/src/probe_manager.rs:231 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
4172024-11-18T18:36:18.190ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
4182024-11-18T18:36:18.198ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_b09b6e80-5341-4ded-a520-3f312403e198/crypt/zone file = sled-storage/src/dataset.rs:303
4192024-11-18T18:36:18.413ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_b09b6e80-5341-4ded-a520-3f312403e198 file = sled-storage/src/dataset.rs:389
4202024-11-18T18:36:18.421ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:440
4212024-11-18T18:36:18.421ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:357
4222024-11-18T18:36:18.421ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:342
4232024-11-18T18:36:18.421ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
4242024-11-18T18:36:18.421ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } path = /scratch/u2_8.vdev
4252024-11-18T18:36:18.431ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_8.vdev file = sled-hardware/src/disk.rs:367
4262024-11-18T18:36:18.431ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = f23dc95a-d65a-4537-b65e-161954e059fa (zpool)
4272024-11-18T18:36:18.554ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: f23dc95a-d65a-4537-b65e-161954e059fa (zpool), kind: External }
4282024-11-18T18:36:18.563ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/dataset.rs:240
4292024-11-18T18:36:18.563ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:242
4302024-11-18T18:36:18.563ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:246
4312024-11-18T18:36:18.563ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:248
4322024-11-18T18:36:18.563ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key")
4332024-11-18T18:36:18.563ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key")
4342024-11-18T18:36:18.563ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
4352024-11-18T18:36:18.608ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
4362024-11-18T18:36:18.617ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_f23dc95a-d65a-4537-b65e-161954e059fa/crypt/zone file = sled-storage/src/dataset.rs:303
4372024-11-18T18:36:18.830ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_f23dc95a-d65a-4537-b65e-161954e059fa file = sled-storage/src/dataset.rs:389
4382024-11-18T18:36:18.838ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:440
4392024-11-18T18:36:18.838ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:357
4402024-11-18T18:36:18.839ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/9216f0f9-29c9-4d33-8c71-0f237e1331a0/config/.omicron-physical-disks.json.tmp file = common/src/ledger.rs:199 request = omicron_physical_disks_ensure
4412024-11-18T18:36:18.839ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 9216f0f9-29c9-4d33-8c71-0f237e1331a0 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: c23ca4a6-dcd8-4835-a278-26664c492a4d (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 0199d4fd-4090-4a7c-83e6-040814eb79bb (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 5840d858-3782-45e1-a6e0-d60d070ad2c3 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 131b4d69-dbb0-4397-9284-176e4a3c973d (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 453f7245-5f70-4fe8-b37e-054e56d5c1c5 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: b09b6e80-5341-4ded-a520-3f312403e198 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: f23dc95a-d65a-4537-b65e-161954e059fa (zpool), kind: External } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
4422024-11-18T18:36:18.839ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
4432024-11-18T18:36:18.839ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
4442024-11-18T18:36:18.839ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/c23ca4a6-dcd8-4835-a278-26664c492a4d/config/.omicron-physical-disks.json.tmp file = common/src/ledger.rs:199 request = omicron_physical_disks_ensure
4452024-11-18T18:36:18.839ZINFOSledAgent: physical disks ensure: Updated storage file = sled-agent/src/sled_agent.rs:888 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
4462024-11-18T18:36:18.839ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
4472024-11-18T18:36:18.839ZINFOSledAgent: physical disks ensure: Propagating new generation of disks file = sled-agent/src/sled_agent.rs:914 generation = Generation(1) sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
4482024-11-18T18:36:18.839ZINFOSledAgent: physical disks ensure: Updated storage monitor file = sled-agent/src/sled_agent.rs:919 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
4492024-11-18T18:36:18.839ZINFOSledAgent: physical disks ensure: Updated zone bundler file = sled-agent/src/sled_agent.rs:924 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
4502024-11-18T18:36:18.839ZINFOSledAgent (ProbeManager): use_only_these_disks: Processing new request file = sled-agent/src/probe_manager.rs:122 gen = Generation(1) sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
4512024-11-18T18:36:18.839ZINFOSledAgent: physical disks ensure: Updated probes file = sled-agent/src/sled_agent.rs:930 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
4522024-11-18T18:36:18.839ZINFOSledAgent (InstanceManager): use_only_these_disks: Processing new request file = sled-agent/src/instance_manager.rs:799 gen = Generation(1)
4532024-11-18T18:36:18.839ZINFOSledAgent: physical disks ensure: Updated instances file = sled-agent/src/sled_agent.rs:935 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
4542024-11-18T18:36:18.839ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 4486372 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:62500 req_id = fc3bd158-1555-4de3-b377-6e4386f45ce3 response_code = 200 uri = /omicron-physical-disks
4552024-11-18T18:36:18.871ZINFOSledAgent (RSS): attempting to set up sled's datasets: DatasetsConfig { generation: Generation(1), datasets: {059196f2-cf97-40fe-a642-74872d8e20c8 (dataset): DatasetConfig { id: 059196f2-cf97-40fe-a642-74872d8e20c8 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0199d4fd-4090-4a7c-83e6-040814eb79bb (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 0f3b3dcf-341e-45af-8826-713a30cb1a0d (dataset): DatasetConfig { id: 0f3b3dcf-341e-45af-8826-713a30cb1a0d (dataset), name: DatasetName { pool_name: ZpoolName { id: b09b6e80-5341-4ded-a520-3f312403e198 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 0f989c0b-7b10-487c-9718-3b491420295f (dataset): DatasetConfig { id: 0f989c0b-7b10-487c-9718-3b491420295f (dataset), name: DatasetName { pool_name: ZpoolName { id: 5840d858-3782-45e1-a6e0-d60d070ad2c3 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 10e788d2-bb32-41ba-8ddd-70c93ef6e5c0 (dataset): DatasetConfig { id: 10e788d2-bb32-41ba-8ddd-70c93ef6e5c0 (dataset), name: DatasetName { pool_name: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 14503785-658f-426b-8357-a429fcc0639f (dataset): DatasetConfig { id: 14503785-658f-426b-8357-a429fcc0639f (dataset), name: DatasetName { pool_name: ZpoolName { id: b09b6e80-5341-4ded-a520-3f312403e198 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 17e19f22-b436-4121-84a3-10407952424f (dataset): DatasetConfig { id: 17e19f22-b436-4121-84a3-10407952424f (dataset), name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 1bc78438-d623-45a7-aa88-efa6d3beef7b (dataset): DatasetConfig { id: 1bc78438-d623-45a7-aa88-efa6d3beef7b (dataset), name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: Clickhouse }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 27192d7f-bbb2-4617-95f7-65a577bf43b0 (dataset): DatasetConfig { id: 27192d7f-bbb2-4617-95f7-65a577bf43b0 (dataset), name: DatasetName { pool_name: ZpoolName { id: b09b6e80-5341-4ded-a520-3f312403e198 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 27442484-b10f-44bf-b933-d2cd5aa7ae1c (dataset): DatasetConfig { id: 27442484-b10f-44bf-b933-d2cd5aa7ae1c (dataset), name: DatasetName { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 278229cd-cb12-4314-81f4-a0a55cb9fd24 (dataset): DatasetConfig { id: 278229cd-cb12-4314-81f4-a0a55cb9fd24 (dataset), name: DatasetName { pool_name: ZpoolName { id: f23dc95a-d65a-4537-b65e-161954e059fa (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2925eb3b-bb3a-4fa0-889f-87ae2ff9ae4b (dataset): DatasetConfig { id: 2925eb3b-bb3a-4fa0-889f-87ae2ff9ae4b (dataset), name: DatasetName { pool_name: ZpoolName { id: 5840d858-3782-45e1-a6e0-d60d070ad2c3 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_b9daa5c1-3421-4e6b-889a-4c4b145361da" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 29b41d61-60b8-4452-85c2-e98de3347b46 (dataset): DatasetConfig { id: 29b41d61-60b8-4452-85c2-e98de3347b46 (dataset), name: DatasetName { pool_name: ZpoolName { id: f23dc95a-d65a-4537-b65e-161954e059fa (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 29fd4423-4349-411e-b21c-e6f5af80b888 (dataset): DatasetConfig { id: 29fd4423-4349-411e-b21c-e6f5af80b888 (dataset), name: DatasetName { pool_name: ZpoolName { id: 453f7245-5f70-4fe8-b37e-054e56d5c1c5 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 2bacf54d-22fc-42a9-a2f6-73f3b93ef8e5 (dataset): DatasetConfig { id: 2bacf54d-22fc-42a9-a2f6-73f3b93ef8e5 (dataset), name: DatasetName { pool_name: ZpoolName { id: 453f7245-5f70-4fe8-b37e-054e56d5c1c5 (zpool), kind: External }, kind: TransientZone { name: "oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2c718e6d-ffbb-441d-8547-e2843527e8a6 (dataset): DatasetConfig { id: 2c718e6d-ffbb-441d-8547-e2843527e8a6 (dataset), name: DatasetName { pool_name: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2d0f116e-a9f3-47cf-996f-6be1bfba8ce8 (dataset): DatasetConfig { id: 2d0f116e-a9f3-47cf-996f-6be1bfba8ce8 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0199d4fd-4090-4a7c-83e6-040814eb79bb (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_d164bfd1-7ae2-47c5-bc53-968633c77a9d" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2d6cf415-4ec8-45c4-9adb-d5848182ef9c (dataset): DatasetConfig { id: 2d6cf415-4ec8-45c4-9adb-d5848182ef9c (dataset), name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: TransientZone { name: "oxz_external_dns_6767b855-63e1-4a7d-ac38-c4e7eb56747f" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 33a943ae-cd57-412b-85a6-d8ec75cd42f8 (dataset): DatasetConfig { id: 33a943ae-cd57-412b-85a6-d8ec75cd42f8 (dataset), name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 3b3149f5-b7ec-4bb2-867c-73a68e4632e5 (dataset): DatasetConfig { id: 3b3149f5-b7ec-4bb2-867c-73a68e4632e5 (dataset), name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: TransientZone { name: "oxz_ntp_6dfcb5a1-e0b1-44a6-97dc-3d4ea97e4f50" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 3cee423f-9bfd-446b-85fc-a1828f03cd07 (dataset): DatasetConfig { id: 3cee423f-9bfd-446b-85fc-a1828f03cd07 (dataset), name: DatasetName { pool_name: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_509f6459-b46e-474b-8247-9cd9c287fa80" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 3fbe6687-10dc-4fd0-91c0-769d11f7a1df (dataset): DatasetConfig { id: 3fbe6687-10dc-4fd0-91c0-769d11f7a1df (dataset), name: DatasetName { pool_name: ZpoolName { id: 0199d4fd-4090-4a7c-83e6-040814eb79bb (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 414b4fbd-dc9c-4d8f-b09d-cecb8d601f0f (dataset): DatasetConfig { id: 414b4fbd-dc9c-4d8f-b09d-cecb8d601f0f (dataset), name: DatasetName { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_efcbb893-1918-43b9-8315-e9bb90d664e8" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 43855bf7-887f-4ba0-9abc-657cbd4db762 (dataset): DatasetConfig { id: 43855bf7-887f-4ba0-9abc-657cbd4db762 (dataset), name: DatasetName { pool_name: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 453d2b51-d6a9-4732-a832-f34f50266abc (dataset): DatasetConfig { id: 453d2b51-d6a9-4732-a832-f34f50266abc (dataset), name: DatasetName { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_07f4fc36-9902-4aeb-a7d9-f99de01ff5c2" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 5e7e7384-89d7-4230-aff4-2dda9771ee31 (dataset): DatasetConfig { id: 5e7e7384-89d7-4230-aff4-2dda9771ee31 (dataset), name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 5e8a0e6b-3eb3-410e-8f78-04cf6bca2b91 (dataset): DatasetConfig { id: 5e8a0e6b-3eb3-410e-8f78-04cf6bca2b91 (dataset), name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: ExternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 6341d350-498e-4853-91ef-ad76173a46f8 (dataset): DatasetConfig { id: 6341d350-498e-4853-91ef-ad76173a46f8 (dataset), name: DatasetName { pool_name: ZpoolName { id: 453f7245-5f70-4fe8-b37e-054e56d5c1c5 (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_d7ddf21e-0e1e-4004-b302-ecb81ba84c09" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 6343e0cf-0360-4532-8733-25a9fbb93542 (dataset): DatasetConfig { id: 6343e0cf-0360-4532-8733-25a9fbb93542 (dataset), name: DatasetName { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 67932ec3-c541-45b6-b59c-60de0d2145ed (dataset): DatasetConfig { id: 67932ec3-c541-45b6-b59c-60de0d2145ed (dataset), name: DatasetName { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 6b50a52a-81c0-4f9f-a1be-66f358422ef2 (dataset): DatasetConfig { id: 6b50a52a-81c0-4f9f-a1be-66f358422ef2 (dataset), name: DatasetName { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, kind: TransientZone { name: "oxz_external_dns_ceea2287-3571-41ea-8f15-b2aeb4a74fd9" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 6eafd3b5-d7b4-4c17-868b-8c30b99e71cd (dataset): DatasetConfig { id: 6eafd3b5-d7b4-4c17-868b-8c30b99e71cd (dataset), name: DatasetName { pool_name: ZpoolName { id: 5840d858-3782-45e1-a6e0-d60d070ad2c3 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 731373af-23cf-4026-8839-e1f94f584509 (dataset): DatasetConfig { id: 731373af-23cf-4026-8839-e1f94f584509 (dataset), name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 76573d98-7e99-4fa1-a6f0-8bae66ee248e (dataset): DatasetConfig { id: 76573d98-7e99-4fa1-a6f0-8bae66ee248e (dataset), name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_9d6a1714-4692-4a2d-9475-dc5b067334ad" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 79831103-b97b-430b-95e9-86ed1105b68b (dataset): DatasetConfig { id: 79831103-b97b-430b-95e9-86ed1105b68b (dataset), name: DatasetName { pool_name: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_b160705f-2e55-4b76-ba79-187b4901f005" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 7a9a7b99-1073-4c1b-bd44-7af4e2a3a381 (dataset): DatasetConfig { id: 7a9a7b99-1073-4c1b-bd44-7af4e2a3a381 (dataset), name: DatasetName { pool_name: ZpoolName { id: 453f7245-5f70-4fe8-b37e-054e56d5c1c5 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 803d7fc5-c027-41b9-9b17-53795541f8e9 (dataset): DatasetConfig { id: 803d7fc5-c027-41b9-9b17-53795541f8e9 (dataset), name: DatasetName { pool_name: ZpoolName { id: 5840d858-3782-45e1-a6e0-d60d070ad2c3 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 80cfebb4-eb3d-45af-ac4d-bb73f2963934 (dataset): DatasetConfig { id: 80cfebb4-eb3d-45af-ac4d-bb73f2963934 (dataset), name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 86511255-812e-4cb0-8980-1ba66a40b0f3 (dataset): DatasetConfig { id: 86511255-812e-4cb0-8980-1ba66a40b0f3 (dataset), name: DatasetName { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 877deec2-af5a-4fee-825f-0fd9b7f55038 (dataset): DatasetConfig { id: 877deec2-af5a-4fee-825f-0fd9b7f55038 (dataset), name: DatasetName { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, kind: ExternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 88e41438-2c11-4b8c-9457-cdc7d829b82d (dataset): DatasetConfig { id: 88e41438-2c11-4b8c-9457-cdc7d829b82d (dataset), name: DatasetName { pool_name: ZpoolName { id: 5840d858-3782-45e1-a6e0-d60d070ad2c3 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_023a09b8-2c2b-483c-be8c-540d1a35ce5a" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 89f6eda3-4975-4a0b-91c7-e36cb1f6ad75 (dataset): DatasetConfig { id: 89f6eda3-4975-4a0b-91c7-e36cb1f6ad75 (dataset), name: DatasetName { pool_name: ZpoolName { id: f23dc95a-d65a-4537-b65e-161954e059fa (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 8bf2e5fa-3e7b-4ca6-8995-3ff170784f01 (dataset): DatasetConfig { id: 8bf2e5fa-3e7b-4ca6-8995-3ff170784f01 (dataset), name: DatasetName { pool_name: ZpoolName { id: 131b4d69-dbb0-4397-9284-176e4a3c973d (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 8cce9518-7800-4366-bda8-755b64f98a66 (dataset): DatasetConfig { id: 8cce9518-7800-4366-bda8-755b64f98a66 (dataset), name: DatasetName { pool_name: ZpoolName { id: 453f7245-5f70-4fe8-b37e-054e56d5c1c5 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 909ae372-e057-4601-bc07-d4be44f76432 (dataset): DatasetConfig { id: 909ae372-e057-4601-bc07-d4be44f76432 (dataset), name: DatasetName { pool_name: ZpoolName { id: 5840d858-3782-45e1-a6e0-d60d070ad2c3 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 95feb982-9d03-47f8-b0a8-e51591700ec1 (dataset): DatasetConfig { id: 95feb982-9d03-47f8-b0a8-e51591700ec1 (dataset), name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: TransientZone { name: "oxz_clickhouse_55ab4bd1-3e97-453c-b90c-176360dea573" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 97a51db2-d6d3-44fc-a45b-fa27aa2e6877 (dataset): DatasetConfig { id: 97a51db2-d6d3-44fc-a45b-fa27aa2e6877 (dataset), name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, ab492327-6151-44bf-8651-0fe14aba0bcd (dataset): DatasetConfig { id: ab492327-6151-44bf-8651-0fe14aba0bcd (dataset), name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_55213fec-a100-46b7-9721-e06d9201c8d5" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, aeb80945-ffaa-41ce-b2e6-f9c1316162b2 (dataset): DatasetConfig { id: aeb80945-ffaa-41ce-b2e6-f9c1316162b2 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0199d4fd-4090-4a7c-83e6-040814eb79bb (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, b171bb77-9113-405e-a169-17e985dd8381 (dataset): DatasetConfig { id: b171bb77-9113-405e-a169-17e985dd8381 (dataset), name: DatasetName { pool_name: ZpoolName { id: 131b4d69-dbb0-4397-9284-176e4a3c973d (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_3aad33dc-2f67-4a47-b973-90de986a5f65" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, b3b281ac-c70c-47f8-92d8-3409bb454c82 (dataset): DatasetConfig { id: b3b281ac-c70c-47f8-92d8-3409bb454c82 (dataset), name: DatasetName { pool_name: ZpoolName { id: 131b4d69-dbb0-4397-9284-176e4a3c973d (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, b844c76d-afcc-4a7d-a4cc-64a8f59029ac (dataset): DatasetConfig { id: b844c76d-afcc-4a7d-a4cc-64a8f59029ac (dataset), name: DatasetName { pool_name: ZpoolName { id: 0199d4fd-4090-4a7c-83e6-040814eb79bb (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, b87ff980-f51b-44b0-9149-8a5c68cede0d (dataset): DatasetConfig { id: b87ff980-f51b-44b0-9149-8a5c68cede0d (dataset), name: DatasetName { pool_name: ZpoolName { id: 5840d858-3782-45e1-a6e0-d60d070ad2c3 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_e782f5fc-076e-414c-b87e-6b09d1bdfaeb" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, b98408b5-8ebd-4578-8f51-687f2bb3278e (dataset): DatasetConfig { id: b98408b5-8ebd-4578-8f51-687f2bb3278e (dataset), name: DatasetName { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, bc696e9f-f0da-45a8-bb28-360eb0917176 (dataset): DatasetConfig { id: bc696e9f-f0da-45a8-bb28-360eb0917176 (dataset), name: DatasetName { pool_name: ZpoolName { id: b09b6e80-5341-4ded-a520-3f312403e198 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_2735d4d6-073c-4860-ae50-e3742cc2cefa" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, becbaee1-dd4a-4554-af88-ef4747d328e5 (dataset): DatasetConfig { id: becbaee1-dd4a-4554-af88-ef4747d328e5 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0199d4fd-4090-4a7c-83e6-040814eb79bb (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, c08b42b2-bc9a-48e2-ae13-df4ebf0c0eb2 (dataset): DatasetConfig { id: c08b42b2-bc9a-48e2-ae13-df4ebf0c0eb2 (dataset), name: DatasetName { pool_name: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_6062ae2a-331c-4cb6-a9b5-eb4310b33530" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, c7f0f536-8e7f-4795-b795-40eef45c2984 (dataset): DatasetConfig { id: c7f0f536-8e7f-4795-b795-40eef45c2984 (dataset), name: DatasetName { pool_name: ZpoolName { id: 453f7245-5f70-4fe8-b37e-054e56d5c1c5 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_81488621-5898-41d4-a112-d3e574fb2d33" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, c9bccd2a-63a7-46ee-b70b-d7ef1644bd61 (dataset): DatasetConfig { id: c9bccd2a-63a7-46ee-b70b-d7ef1644bd61 (dataset), name: DatasetName { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, cbddf414-8862-4e44-891d-d56adca4799d (dataset): DatasetConfig { id: cbddf414-8862-4e44-891d-d56adca4799d (dataset), name: DatasetName { pool_name: ZpoolName { id: f23dc95a-d65a-4537-b65e-161954e059fa (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_10904f20-d090-41dc-9a54-dab632f01d4a" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d43e44a1-acbd-48d7-bb21-3d48cbb26a3d (dataset): DatasetConfig { id: d43e44a1-acbd-48d7-bb21-3d48cbb26a3d (dataset), name: DatasetName { pool_name: ZpoolName { id: 0199d4fd-4090-4a7c-83e6-040814eb79bb (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d8d76285-7824-44f0-a60e-fe61bb1ea5a1 (dataset): DatasetConfig { id: d8d76285-7824-44f0-a60e-fe61bb1ea5a1 (dataset), name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, db472472-36c4-49e6-8087-93894660846c (dataset): DatasetConfig { id: db472472-36c4-49e6-8087-93894660846c (dataset), name: DatasetName { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, e16144c5-325f-4faa-8011-822489587cd8 (dataset): DatasetConfig { id: e16144c5-325f-4faa-8011-822489587cd8 (dataset), name: DatasetName { pool_name: ZpoolName { id: 131b4d69-dbb0-4397-9284-176e4a3c973d (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_1fa25dd7-d8f9-49ef-bb20-c6bad413aace" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, e37ce29b-2978-4e94-a120-a4bc43a88c20 (dataset): DatasetConfig { id: e37ce29b-2978-4e94-a120-a4bc43a88c20 (dataset), name: DatasetName { pool_name: ZpoolName { id: 131b4d69-dbb0-4397-9284-176e4a3c973d (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ee20377f-6181-41a5-9709-0db033ef6a05 (dataset): DatasetConfig { id: ee20377f-6181-41a5-9709-0db033ef6a05 (dataset), name: DatasetName { pool_name: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, f9546b32-4b5d-4d16-bd28-572be3a688ba (dataset): DatasetConfig { id: f9546b32-4b5d-4d16-bd28-572be3a688ba (dataset), name: DatasetName { pool_name: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }} } file = sled-agent/src/rack_setup/service.rs:457 sled_address = [fd00:1122:3344:101::1]:12345
4562024-11-18T18:36:18.872ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:56563
4572024-11-18T18:36:18.872ZINFOSledAgent: datasets ensure file = sled-agent/src/sled_agent.rs:850 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
4582024-11-18T18:36:18.872ZINFOSledAgent (StorageManager): Received DatasetsEnsure { config: DatasetsConfig { generation: Generation(1), datasets: {059196f2-cf97-40fe-a642-74872d8e20c8 (dataset): DatasetConfig { id: 059196f2-cf97-40fe-a642-74872d8e20c8 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0199d4fd-4090-4a7c-83e6-040814eb79bb (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 0f3b3dcf-341e-45af-8826-713a30cb1a0d (dataset): DatasetConfig { id: 0f3b3dcf-341e-45af-8826-713a30cb1a0d (dataset), name: DatasetName { pool_name: ZpoolName { id: b09b6e80-5341-4ded-a520-3f312403e198 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 0f989c0b-7b10-487c-9718-3b491420295f (dataset): DatasetConfig { id: 0f989c0b-7b10-487c-9718-3b491420295f (dataset), name: DatasetName { pool_name: ZpoolName { id: 5840d858-3782-45e1-a6e0-d60d070ad2c3 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 10e788d2-bb32-41ba-8ddd-70c93ef6e5c0 (dataset): DatasetConfig { id: 10e788d2-bb32-41ba-8ddd-70c93ef6e5c0 (dataset), name: DatasetName { pool_name: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 14503785-658f-426b-8357-a429fcc0639f (dataset): DatasetConfig { id: 14503785-658f-426b-8357-a429fcc0639f (dataset), name: DatasetName { pool_name: ZpoolName { id: b09b6e80-5341-4ded-a520-3f312403e198 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 17e19f22-b436-4121-84a3-10407952424f (dataset): DatasetConfig { id: 17e19f22-b436-4121-84a3-10407952424f (dataset), name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 1bc78438-d623-45a7-aa88-efa6d3beef7b (dataset): DatasetConfig { id: 1bc78438-d623-45a7-aa88-efa6d3beef7b (dataset), name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: Clickhouse }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 27192d7f-bbb2-4617-95f7-65a577bf43b0 (dataset): DatasetConfig { id: 27192d7f-bbb2-4617-95f7-65a577bf43b0 (dataset), name: DatasetName { pool_name: ZpoolName { id: b09b6e80-5341-4ded-a520-3f312403e198 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 27442484-b10f-44bf-b933-d2cd5aa7ae1c (dataset): DatasetConfig { id: 27442484-b10f-44bf-b933-d2cd5aa7ae1c (dataset), name: DatasetName { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 278229cd-cb12-4314-81f4-a0a55cb9fd24 (dataset): DatasetConfig { id: 278229cd-cb12-4314-81f4-a0a55cb9fd24 (dataset), name: DatasetName { pool_name: ZpoolName { id: f23dc95a-d65a-4537-b65e-161954e059fa (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2925eb3b-bb3a-4fa0-889f-87ae2ff9ae4b (dataset): DatasetConfig { id: 2925eb3b-bb3a-4fa0-889f-87ae2ff9ae4b (dataset), name: DatasetName { pool_name: ZpoolName { id: 5840d858-3782-45e1-a6e0-d60d070ad2c3 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_b9daa5c1-3421-4e6b-889a-4c4b145361da" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 29b41d61-60b8-4452-85c2-e98de3347b46 (dataset): DatasetConfig { id: 29b41d61-60b8-4452-85c2-e98de3347b46 (dataset), name: DatasetName { pool_name: ZpoolName { id: f23dc95a-d65a-4537-b65e-161954e059fa (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 29fd4423-4349-411e-b21c-e6f5af80b888 (dataset): DatasetConfig { id: 29fd4423-4349-411e-b21c-e6f5af80b888 (dataset), name: DatasetName { pool_name: ZpoolName { id: 453f7245-5f70-4fe8-b37e-054e56d5c1c5 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 2bacf54d-22fc-42a9-a2f6-73f3b93ef8e5 (dataset): DatasetConfig { id: 2bacf54d-22fc-42a9-a2f6-73f3b93ef8e5 (dataset), name: DatasetName { pool_name: ZpoolName { id: 453f7245-5f70-4fe8-b37e-054e56d5c1c5 (zpool), kind: External }, kind: TransientZone { name: "oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2c718e6d-ffbb-441d-8547-e2843527e8a6 (dataset): DatasetConfig { id: 2c718e6d-ffbb-441d-8547-e2843527e8a6 (dataset), name: DatasetName { pool_name: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2d0f116e-a9f3-47cf-996f-6be1bfba8ce8 (dataset): DatasetConfig { id: 2d0f116e-a9f3-47cf-996f-6be1bfba8ce8 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0199d4fd-4090-4a7c-83e6-040814eb79bb (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_d164bfd1-7ae2-47c5-bc53-968633c77a9d" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2d6cf415-4ec8-45c4-9adb-d5848182ef9c (dataset): DatasetConfig { id: 2d6cf415-4ec8-45c4-9adb-d5848182ef9c (dataset), name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: TransientZone { name: "oxz_external_dns_6767b855-63e1-4a7d-ac38-c4e7eb56747f" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 33a943ae-cd57-412b-85a6-d8ec75cd42f8 (dataset): DatasetConfig { id: 33a943ae-cd57-412b-85a6-d8ec75cd42f8 (dataset), name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 3b3149f5-b7ec-4bb2-867c-73a68e4632e5 (dataset): DatasetConfig { id: 3b3149f5-b7ec-4bb2-867c-73a68e4632e5 (dataset), name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: TransientZone { name: "oxz_ntp_6dfcb5a1-e0b1-44a6-97dc-3d4ea97e4f50" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 3cee423f-9bfd-446b-85fc-a1828f03cd07 (dataset): DatasetConfig { id: 3cee423f-9bfd-446b-85fc-a1828f03cd07 (dataset), name: DatasetName { pool_name: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_509f6459-b46e-474b-8247-9cd9c287fa80" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 3fbe6687-10dc-4fd0-91c0-769d11f7a1df (dataset): DatasetConfig { id: 3fbe6687-10dc-4fd0-91c0-769d11f7a1df (dataset), name: DatasetName { pool_name: ZpoolName { id: 0199d4fd-4090-4a7c-83e6-040814eb79bb (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 414b4fbd-dc9c-4d8f-b09d-cecb8d601f0f (dataset): DatasetConfig { id: 414b4fbd-dc9c-4d8f-b09d-cecb8d601f0f (dataset), name: DatasetName { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_efcbb893-1918-43b9-8315-e9bb90d664e8" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 43855bf7-887f-4ba0-9abc-657cbd4db762 (dataset): DatasetConfig { id: 43855bf7-887f-4ba0-9abc-657cbd4db762 (dataset), name: DatasetName { pool_name: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 453d2b51-d6a9-4732-a832-f34f50266abc (dataset): DatasetConfig { id: 453d2b51-d6a9-4732-a832-f34f50266abc (dataset), name: DatasetName { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_07f4fc36-9902-4aeb-a7d9-f99de01ff5c2" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 5e7e7384-89d7-4230-aff4-2dda9771ee31 (dataset): DatasetConfig { id: 5e7e7384-89d7-4230-aff4-2dda9771ee31 (dataset), name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 5e8a0e6b-3eb3-410e-8f78-04cf6bca2b91 (dataset): DatasetConfig { id: 5e8a0e6b-3eb3-410e-8f78-04cf6bca2b91 (dataset), name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: ExternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 6341d350-498e-4853-91ef-ad76173a46f8 (dataset): DatasetConfig { id: 6341d350-498e-4853-91ef-ad76173a46f8 (dataset), name: DatasetName { pool_name: ZpoolName { id: 453f7245-5f70-4fe8-b37e-054e56d5c1c5 (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_d7ddf21e-0e1e-4004-b302-ecb81ba84c09" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 6343e0cf-0360-4532-8733-25a9fbb93542 (dataset): DatasetConfig { id: 6343e0cf-0360-4532-8733-25a9fbb93542 (dataset), name: DatasetName { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 67932ec3-c541-45b6-b59c-60de0d2145ed (dataset): DatasetConfig { id: 67932ec3-c541-45b6-b59c-60de0d2145ed (dataset), name: DatasetName { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 6b50a52a-81c0-4f9f-a1be-66f358422ef2 (dataset): DatasetConfig { id: 6b50a52a-81c0-4f9f-a1be-66f358422ef2 (dataset), name: DatasetName { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, kind: TransientZone { name: "oxz_external_dns_ceea2287-3571-41ea-8f15-b2aeb4a74fd9" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 6eafd3b5-d7b4-4c17-868b-8c30b99e71cd (dataset): DatasetConfig { id: 6eafd3b5-d7b4-4c17-868b-8c30b99e71cd (dataset), name: DatasetName { pool_name: ZpoolName { id: 5840d858-3782-45e1-a6e0-d60d070ad2c3 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 731373af-23cf-4026-8839-e1f94f584509 (dataset): DatasetConfig { id: 731373af-23cf-4026-8839-e1f94f584509 (dataset), name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 76573d98-7e99-4fa1-a6f0-8bae66ee248e (dataset): DatasetConfig { id: 76573d98-7e99-4fa1-a6f0-8bae66ee248e (dataset), name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_9d6a1714-4692-4a2d-9475-dc5b067334ad" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 79831103-b97b-430b-95e9-86ed1105b68b (dataset): DatasetConfig { id: 79831103-b97b-430b-95e9-86ed1105b68b (dataset), name: DatasetName { pool_name: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_b160705f-2e55-4b76-ba79-187b4901f005" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 7a9a7b99-1073-4c1b-bd44-7af4e2a3a381 (dataset): DatasetConfig { id: 7a9a7b99-1073-4c1b-bd44-7af4e2a3a381 (dataset), name: DatasetName { pool_name: ZpoolName { id: 453f7245-5f70-4fe8-b37e-054e56d5c1c5 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 803d7fc5-c027-41b9-9b17-53795541f8e9 (dataset): DatasetConfig { id: 803d7fc5-c027-41b9-9b17-53795541f8e9 (dataset), name: DatasetName { pool_name: ZpoolName { id: 5840d858-3782-45e1-a6e0-d60d070ad2c3 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 80cfebb4-eb3d-45af-ac4d-bb73f2963934 (dataset): DatasetConfig { id: 80cfebb4-eb3d-45af-ac4d-bb73f2963934 (dataset), name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 86511255-812e-4cb0-8980-1ba66a40b0f3 (dataset): DatasetConfig { id: 86511255-812e-4cb0-8980-1ba66a40b0f3 (dataset), name: DatasetName { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 877deec2-af5a-4fee-825f-0fd9b7f55038 (dataset): DatasetConfig { id: 877deec2-af5a-4fee-825f-0fd9b7f55038 (dataset), name: DatasetName { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, kind: ExternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 88e41438-2c11-4b8c-9457-cdc7d829b82d (dataset): DatasetConfig { id: 88e41438-2c11-4b8c-9457-cdc7d829b82d (dataset), name: DatasetName { pool_name: ZpoolName { id: 5840d858-3782-45e1-a6e0-d60d070ad2c3 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_023a09b8-2c2b-483c-be8c-540d1a35ce5a" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 89f6eda3-4975-4a0b-91c7-e36cb1f6ad75 (dataset): DatasetConfig { id: 89f6eda3-4975-4a0b-91c7-e36cb1f6ad75 (dataset), name: DatasetName { pool_name: ZpoolName { id: f23dc95a-d65a-4537-b65e-161954e059fa (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 8bf2e5fa-3e7b-4ca6-8995-3ff170784f01 (dataset): DatasetConfig { id: 8bf2e5fa-3e7b-4ca6-8995-3ff170784f01 (dataset), name: DatasetName { pool_name: ZpoolName { id: 131b4d69-dbb0-4397-9284-176e4a3c973d (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 8cce9518-7800-4366-bda8-755b64f98a66 (dataset): DatasetConfig { id: 8cce9518-7800-4366-bda8-755b64f98a66 (dataset), name: DatasetName { pool_name: ZpoolName { id: 453f7245-5f70-4fe8-b37e-054e56d5c1c5 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 909ae372-e057-4601-bc07-d4be44f76432 (dataset): DatasetConfig { id: 909ae372-e057-4601-bc07-d4be44f76432 (dataset), name: DatasetName { pool_name: ZpoolName { id: 5840d858-3782-45e1-a6e0-d60d070ad2c3 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 95feb982-9d03-47f8-b0a8-e51591700ec1 (dataset): DatasetConfig { id: 95feb982-9d03-47f8-b0a8-e51591700ec1 (dataset), name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: TransientZone { name: "oxz_clickhouse_55ab4bd1-3e97-453c-b90c-176360dea573" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 97a51db2-d6d3-44fc-a45b-fa27aa2e6877 (dataset): DatasetConfig { id: 97a51db2-d6d3-44fc-a45b-fa27aa2e6877 (dataset), name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, ab492327-6151-44bf-8651-0fe14aba0bcd (dataset): DatasetConfig { id: ab492327-6151-44bf-8651-0fe14aba0bcd (dataset), name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_55213fec-a100-46b7-9721-e06d9201c8d5" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, aeb80945-ffaa-41ce-b2e6-f9c1316162b2 (dataset): DatasetConfig { id: aeb80945-ffaa-41ce-b2e6-f9c1316162b2 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0199d4fd-4090-4a7c-83e6-040814eb79bb (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, b171bb77-9113-405e-a169-17e985dd8381 (dataset): DatasetConfig { id: b171bb77-9113-405e-a169-17e985dd8381 (dataset), name: DatasetName { pool_name: ZpoolName { id: 131b4d69-dbb0-4397-9284-176e4a3c973d (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_3aad33dc-2f67-4a47-b973-90de986a5f65" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, b3b281ac-c70c-47f8-92d8-3409bb454c82 (dataset): DatasetConfig { id: b3b281ac-c70c-47f8-92d8-3409bb454c82 (dataset), name: DatasetName { pool_name: ZpoolName { id: 131b4d69-dbb0-4397-9284-176e4a3c973d (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, b844c76d-afcc-4a7d-a4cc-64a8f59029ac (dataset): DatasetConfig { id: b844c76d-afcc-4a7d-a4cc-64a8f59029ac (dataset), name: DatasetName { pool_name: ZpoolName { id: 0199d4fd-4090-4a7c-83e6-040814eb79bb (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, b87ff980-f51b-44b0-9149-8a5c68cede0d (dataset): DatasetConfig { id: b87ff980-f51b-44b0-9149-8a5c68cede0d (dataset), name: DatasetName { pool_name: ZpoolName { id: 5840d858-3782-45e1-a6e0-d60d070ad2c3 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_e782f5fc-076e-414c-b87e-6b09d1bdfaeb" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, b98408b5-8ebd-4578-8f51-687f2bb3278e (dataset): DatasetConfig { id: b98408b5-8ebd-4578-8f51-687f2bb3278e (dataset), name: DatasetName { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, bc696e9f-f0da-45a8-bb28-360eb0917176 (dataset): DatasetConfig { id: bc696e9f-f0da-45a8-bb28-360eb0917176 (dataset), name: DatasetName { pool_name: ZpoolName { id: b09b6e80-5341-4ded-a520-3f312403e198 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_2735d4d6-073c-4860-ae50-e3742cc2cefa" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, becbaee1-dd4a-4554-af88-ef4747d328e5 (dataset): DatasetConfig { id: becbaee1-dd4a-4554-af88-ef4747d328e5 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0199d4fd-4090-4a7c-83e6-040814eb79bb (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, c08b42b2-bc9a-48e2-ae13-df4ebf0c0eb2 (dataset): DatasetConfig { id: c08b42b2-bc9a-48e2-ae13-df4ebf0c0eb2 (dataset), name: DatasetName { pool_name: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_6062ae2a-331c-4cb6-a9b5-eb4310b33530" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, c7f0f536-8e7f-4795-b795-40eef45c2984 (dataset): DatasetConfig { id: c7f0f536-8e7f-4795-b795-40eef45c2984 (dataset), name: DatasetName { pool_name: ZpoolName { id: 453f7245-5f70-4fe8-b37e-054e56d5c1c5 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_81488621-5898-41d4-a112-d3e574fb2d33" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, c9bccd2a-63a7-46ee-b70b-d7ef1644bd61 (dataset): DatasetConfig { id: c9bccd2a-63a7-46ee-b70b-d7ef1644bd61 (dataset), name: DatasetName { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, cbddf414-8862-4e44-891d-d56adca4799d (dataset): DatasetConfig { id: cbddf414-8862-4e44-891d-d56adca4799d (dataset), name: DatasetName { pool_name: ZpoolName { id: f23dc95a-d65a-4537-b65e-161954e059fa (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_10904f20-d090-41dc-9a54-dab632f01d4a" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d43e44a1-acbd-48d7-bb21-3d48cbb26a3d (dataset): DatasetConfig { id: d43e44a1-acbd-48d7-bb21-3d48cbb26a3d (dataset), name: DatasetName { pool_name: ZpoolName { id: 0199d4fd-4090-4a7c-83e6-040814eb79bb (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d8d76285-7824-44f0-a60e-fe61bb1ea5a1 (dataset): DatasetConfig { id: d8d76285-7824-44f0-a60e-fe61bb1ea5a1 (dataset), name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, db472472-36c4-49e6-8087-93894660846c (dataset): DatasetConfig { id: db472472-36c4-49e6-8087-93894660846c (dataset), name: DatasetName { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, e16144c5-325f-4faa-8011-822489587cd8 (dataset): DatasetConfig { id: e16144c5-325f-4faa-8011-822489587cd8 (dataset), name: DatasetName { pool_name: ZpoolName { id: 131b4d69-dbb0-4397-9284-176e4a3c973d (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_1fa25dd7-d8f9-49ef-bb20-c6bad413aace" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, e37ce29b-2978-4e94-a120-a4bc43a88c20 (dataset): DatasetConfig { id: e37ce29b-2978-4e94-a120-a4bc43a88c20 (dataset), name: DatasetName { pool_name: ZpoolName { id: 131b4d69-dbb0-4397-9284-176e4a3c973d (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ee20377f-6181-41a5-9709-0db033ef6a05 (dataset): DatasetConfig { id: ee20377f-6181-41a5-9709-0db033ef6a05 (dataset), name: DatasetName { pool_name: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, f9546b32-4b5d-4d16-bd28-572be3a688ba (dataset): DatasetConfig { id: f9546b32-4b5d-4d16-bd28-572be3a688ba (dataset), name: DatasetName { pool_name: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }} }, tx: ... } file = sled-storage/src/manager.rs:603
4592024-11-18T18:36:18.872ZINFOSledAgent (StorageManager): No ledger in /pool/int/9216f0f9-29c9-4d33-8c71-0f237e1331a0/config/omicron-datasets.json file = common/src/ledger.rs:188 request = datasets_ensure
4602024-11-18T18:36:18.872ZINFOSledAgent (StorageManager): No ledger in /pool/int/c23ca4a6-dcd8-4835-a278-26664c492a4d/config/omicron-datasets.json file = common/src/ledger.rs:188 request = datasets_ensure
4612024-11-18T18:36:18.872ZINFOSledAgent (StorageManager): No previously-stored 'requested datasets', creating new ledger file = sled-storage/src/manager.rs:905 request = datasets_ensure
4622024-11-18T18:36:18.872ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4632024-11-18T18:36:18.872ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_0199d4fd-4090-4a7c-83e6-040814eb79bb/crucible" } file = sled-storage/src/manager.rs:1331
4642024-11-18T18:36:18.983ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4652024-11-18T18:36:18.983ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_b09b6e80-5341-4ded-a520-3f312403e198/crucible" } file = sled-storage/src/manager.rs:1331
4662024-11-18T18:36:19.092ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4672024-11-18T18:36:19.092ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/5840d858-3782-45e1-a6e0-d60d070ad2c3/crypt/zone"), full_name: "oxp_5840d858-3782-45e1-a6e0-d60d070ad2c3/crypt/zone" } file = sled-storage/src/manager.rs:1331
4682024-11-18T18:36:19.151ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/ee2b3f44-b73f-4358-9df7-edb19f5079e2) file = sled-agent/src/probe_manager.rs:231 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
4692024-11-18T18:36:19.174ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4702024-11-18T18:36:19.175ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_df4faf1b-f747-4cda-8481-deaa8523a533/crypt/internal_dns" } file = sled-storage/src/manager.rs:1331
4712024-11-18T18:36:19.287ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4722024-11-18T18:36:19.287ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/b09b6e80-5341-4ded-a520-3f312403e198/crypt/zone"), full_name: "oxp_b09b6e80-5341-4ded-a520-3f312403e198/crypt/zone" } file = sled-storage/src/manager.rs:1331
4732024-11-18T18:36:19.370ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4742024-11-18T18:36:19.370ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/4be98a0c-ab4c-4309-948b-04d776603876/crypt/zone"), full_name: "oxp_4be98a0c-ab4c-4309-948b-04d776603876/crypt/zone" } file = sled-storage/src/manager.rs:1331
4752024-11-18T18:36:19.458ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4762024-11-18T18:36:19.458ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_4be98a0c-ab4c-4309-948b-04d776603876/crypt/clickhouse" } file = sled-storage/src/manager.rs:1331
4772024-11-18T18:36:19.577ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4782024-11-18T18:36:19.577ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/b09b6e80-5341-4ded-a520-3f312403e198/crypt/debug"), full_name: "oxp_b09b6e80-5341-4ded-a520-3f312403e198/crypt/debug" } file = sled-storage/src/manager.rs:1331
4792024-11-18T18:36:19.658ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4802024-11-18T18:36:19.658ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_4f35fd7e-936a-4e09-946b-7cb174580897/crucible" } file = sled-storage/src/manager.rs:1331
4812024-11-18T18:36:19.762ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4822024-11-18T18:36:19.762ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_f23dc95a-d65a-4537-b65e-161954e059fa/crucible" } file = sled-storage/src/manager.rs:1331
4832024-11-18T18:36:19.849ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4842024-11-18T18:36:19.849ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/5840d858-3782-45e1-a6e0-d60d070ad2c3/crypt/zone/oxz_cockroachdb_b9daa5c1-3421-4e6b-889a-4c4b145361da"), full_name: "oxp_5840d858-3782-45e1-a6e0-d60d070ad2c3/crypt/zone/oxz_cockroachdb_b9daa5c1-3421-4e6b-889a-4c4b145361da" } file = sled-storage/src/manager.rs:1331
4852024-11-18T18:36:19.957ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4862024-11-18T18:36:19.957ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/f23dc95a-d65a-4537-b65e-161954e059fa/crypt/zone"), full_name: "oxp_f23dc95a-d65a-4537-b65e-161954e059fa/crypt/zone" } file = sled-storage/src/manager.rs:1331
4872024-11-18T18:36:20.033ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4882024-11-18T18:36:20.033ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/453f7245-5f70-4fe8-b37e-054e56d5c1c5/crypt/debug"), full_name: "oxp_453f7245-5f70-4fe8-b37e-054e56d5c1c5/crypt/debug" } file = sled-storage/src/manager.rs:1331
4892024-11-18T18:36:20.100ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4902024-11-18T18:36:20.100ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/453f7245-5f70-4fe8-b37e-054e56d5c1c5/crypt/zone/oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6"), full_name: "oxp_453f7245-5f70-4fe8-b37e-054e56d5c1c5/crypt/zone/oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6" } file = sled-storage/src/manager.rs:1331
4912024-11-18T18:36:20.155ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/ee2b3f44-b73f-4358-9df7-edb19f5079e2) file = sled-agent/src/probe_manager.rs:231 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
4922024-11-18T18:36:20.206ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4932024-11-18T18:36:20.206ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_df4faf1b-f747-4cda-8481-deaa8523a533/crypt/cockroachdb" } file = sled-storage/src/manager.rs:1331
4942024-11-18T18:36:20.314ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4952024-11-18T18:36:20.314ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/0199d4fd-4090-4a7c-83e6-040814eb79bb/crypt/zone/oxz_cockroachdb_d164bfd1-7ae2-47c5-bc53-968633c77a9d"), full_name: "oxp_0199d4fd-4090-4a7c-83e6-040814eb79bb/crypt/zone/oxz_cockroachdb_d164bfd1-7ae2-47c5-bc53-968633c77a9d" } file = sled-storage/src/manager.rs:1331
4962024-11-18T18:36:20.425ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4972024-11-18T18:36:20.425ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/4be98a0c-ab4c-4309-948b-04d776603876/crypt/zone/oxz_external_dns_6767b855-63e1-4a7d-ac38-c4e7eb56747f"), full_name: "oxp_4be98a0c-ab4c-4309-948b-04d776603876/crypt/zone/oxz_external_dns_6767b855-63e1-4a7d-ac38-c4e7eb56747f" } file = sled-storage/src/manager.rs:1331
4982024-11-18T18:36:20.538ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4992024-11-18T18:36:20.538ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_4be98a0c-ab4c-4309-948b-04d776603876/crypt/internal_dns" } file = sled-storage/src/manager.rs:1331
5002024-11-18T18:36:20.639ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5012024-11-18T18:36:20.639ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/4be98a0c-ab4c-4309-948b-04d776603876/crypt/zone/oxz_ntp_6dfcb5a1-e0b1-44a6-97dc-3d4ea97e4f50"), full_name: "oxp_4be98a0c-ab4c-4309-948b-04d776603876/crypt/zone/oxz_ntp_6dfcb5a1-e0b1-44a6-97dc-3d4ea97e4f50" } file = sled-storage/src/manager.rs:1331
5022024-11-18T18:36:20.753ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5032024-11-18T18:36:20.753ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/df4faf1b-f747-4cda-8481-deaa8523a533/crypt/zone/oxz_cockroachdb_509f6459-b46e-474b-8247-9cd9c287fa80"), full_name: "oxp_df4faf1b-f747-4cda-8481-deaa8523a533/crypt/zone/oxz_cockroachdb_509f6459-b46e-474b-8247-9cd9c287fa80" } file = sled-storage/src/manager.rs:1331
5042024-11-18T18:36:20.863ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5052024-11-18T18:36:20.863ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_0199d4fd-4090-4a7c-83e6-040814eb79bb/crypt/cockroachdb" } file = sled-storage/src/manager.rs:1331
5062024-11-18T18:36:20.966ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5072024-11-18T18:36:20.966ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/4f35fd7e-936a-4e09-946b-7cb174580897/crypt/zone/oxz_crucible_efcbb893-1918-43b9-8315-e9bb90d664e8"), full_name: "oxp_4f35fd7e-936a-4e09-946b-7cb174580897/crypt/zone/oxz_crucible_efcbb893-1918-43b9-8315-e9bb90d664e8" } file = sled-storage/src/manager.rs:1331
5082024-11-18T18:36:21.075ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5092024-11-18T18:36:21.075ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/df4faf1b-f747-4cda-8481-deaa8523a533/crypt/debug"), full_name: "oxp_df4faf1b-f747-4cda-8481-deaa8523a533/crypt/debug" } file = sled-storage/src/manager.rs:1331
5102024-11-18T18:36:21.153ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5112024-11-18T18:36:21.153ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/4f35fd7e-936a-4e09-946b-7cb174580897/crypt/zone/oxz_internal_dns_07f4fc36-9902-4aeb-a7d9-f99de01ff5c2"), full_name: "oxp_4f35fd7e-936a-4e09-946b-7cb174580897/crypt/zone/oxz_internal_dns_07f4fc36-9902-4aeb-a7d9-f99de01ff5c2" } file = sled-storage/src/manager.rs:1331
5122024-11-18T18:36:21.158ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/ee2b3f44-b73f-4358-9df7-edb19f5079e2) file = sled-agent/src/probe_manager.rs:231 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
5132024-11-18T18:36:21.269ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5142024-11-18T18:36:21.269ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/4be98a0c-ab4c-4309-948b-04d776603876/crypt/zone/oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1"), full_name: "oxp_4be98a0c-ab4c-4309-948b-04d776603876/crypt/zone/oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1" } file = sled-storage/src/manager.rs:1331
5152024-11-18T18:36:21.383ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5162024-11-18T18:36:21.383ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_4be98a0c-ab4c-4309-948b-04d776603876/crypt/external_dns" } file = sled-storage/src/manager.rs:1331
5172024-11-18T18:36:21.494ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5182024-11-18T18:36:21.494ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/453f7245-5f70-4fe8-b37e-054e56d5c1c5/crypt/zone/oxz_nexus_d7ddf21e-0e1e-4004-b302-ecb81ba84c09"), full_name: "oxp_453f7245-5f70-4fe8-b37e-054e56d5c1c5/crypt/zone/oxz_nexus_d7ddf21e-0e1e-4004-b302-ecb81ba84c09" } file = sled-storage/src/manager.rs:1331
5192024-11-18T18:36:21.614ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5202024-11-18T18:36:21.614ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_4f35fd7e-936a-4e09-946b-7cb174580897/crypt/internal_dns" } file = sled-storage/src/manager.rs:1331
5212024-11-18T18:36:21.725ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5222024-11-18T18:36:21.725ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/4f35fd7e-936a-4e09-946b-7cb174580897/crypt/zone"), full_name: "oxp_4f35fd7e-936a-4e09-946b-7cb174580897/crypt/zone" } file = sled-storage/src/manager.rs:1331
5232024-11-18T18:36:21.804ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5242024-11-18T18:36:21.804ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/4f35fd7e-936a-4e09-946b-7cb174580897/crypt/zone/oxz_external_dns_ceea2287-3571-41ea-8f15-b2aeb4a74fd9"), full_name: "oxp_4f35fd7e-936a-4e09-946b-7cb174580897/crypt/zone/oxz_external_dns_ceea2287-3571-41ea-8f15-b2aeb4a74fd9" } file = sled-storage/src/manager.rs:1331
5252024-11-18T18:36:21.919ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5262024-11-18T18:36:21.919ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_5840d858-3782-45e1-a6e0-d60d070ad2c3/crucible" } file = sled-storage/src/manager.rs:1331
5272024-11-18T18:36:22.028ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5282024-11-18T18:36:22.028ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_4be98a0c-ab4c-4309-948b-04d776603876/crucible" } file = sled-storage/src/manager.rs:1331
5292024-11-18T18:36:22.138ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5302024-11-18T18:36:22.138ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/4be98a0c-ab4c-4309-948b-04d776603876/crypt/zone/oxz_cockroachdb_9d6a1714-4692-4a2d-9475-dc5b067334ad"), full_name: "oxp_4be98a0c-ab4c-4309-948b-04d776603876/crypt/zone/oxz_cockroachdb_9d6a1714-4692-4a2d-9475-dc5b067334ad" } file = sled-storage/src/manager.rs:1331
5312024-11-18T18:36:22.159ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/ee2b3f44-b73f-4358-9df7-edb19f5079e2) file = sled-agent/src/probe_manager.rs:231 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
5322024-11-18T18:36:22.255ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5332024-11-18T18:36:22.255ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/df4faf1b-f747-4cda-8481-deaa8523a533/crypt/zone/oxz_internal_dns_b160705f-2e55-4b76-ba79-187b4901f005"), full_name: "oxp_df4faf1b-f747-4cda-8481-deaa8523a533/crypt/zone/oxz_internal_dns_b160705f-2e55-4b76-ba79-187b4901f005" } file = sled-storage/src/manager.rs:1331
5342024-11-18T18:36:22.373ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5352024-11-18T18:36:22.373ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_453f7245-5f70-4fe8-b37e-054e56d5c1c5/crucible" } file = sled-storage/src/manager.rs:1331
5362024-11-18T18:36:22.482ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5372024-11-18T18:36:22.482ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_5840d858-3782-45e1-a6e0-d60d070ad2c3/crypt/cockroachdb" } file = sled-storage/src/manager.rs:1331
5382024-11-18T18:36:22.590ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5392024-11-18T18:36:22.590ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/4be98a0c-ab4c-4309-948b-04d776603876/crypt/zone/oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34"), full_name: "oxp_4be98a0c-ab4c-4309-948b-04d776603876/crypt/zone/oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34" } file = sled-storage/src/manager.rs:1331
5402024-11-18T18:36:22.734ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5412024-11-18T18:36:22.734ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/4f35fd7e-936a-4e09-946b-7cb174580897/crypt/debug"), full_name: "oxp_4f35fd7e-936a-4e09-946b-7cb174580897/crypt/debug" } file = sled-storage/src/manager.rs:1331
5422024-11-18T18:36:22.815ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5432024-11-18T18:36:22.815ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_4f35fd7e-936a-4e09-946b-7cb174580897/crypt/external_dns" } file = sled-storage/src/manager.rs:1331
5442024-11-18T18:36:22.926ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5452024-11-18T18:36:22.926ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/5840d858-3782-45e1-a6e0-d60d070ad2c3/crypt/zone/oxz_crucible_pantry_023a09b8-2c2b-483c-be8c-540d1a35ce5a"), full_name: "oxp_5840d858-3782-45e1-a6e0-d60d070ad2c3/crypt/zone/oxz_crucible_pantry_023a09b8-2c2b-483c-be8c-540d1a35ce5a" } file = sled-storage/src/manager.rs:1331
5462024-11-18T18:36:23.044ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5472024-11-18T18:36:23.044ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/f23dc95a-d65a-4537-b65e-161954e059fa/crypt/debug"), full_name: "oxp_f23dc95a-d65a-4537-b65e-161954e059fa/crypt/debug" } file = sled-storage/src/manager.rs:1331
5482024-11-18T18:36:23.127ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5492024-11-18T18:36:23.127ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/131b4d69-dbb0-4397-9284-176e4a3c973d/crypt/debug"), full_name: "oxp_131b4d69-dbb0-4397-9284-176e4a3c973d/crypt/debug" } file = sled-storage/src/manager.rs:1331
5502024-11-18T18:36:23.162ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/ee2b3f44-b73f-4358-9df7-edb19f5079e2) file = sled-agent/src/probe_manager.rs:231 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
5512024-11-18T18:36:23.209ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5522024-11-18T18:36:23.209ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/453f7245-5f70-4fe8-b37e-054e56d5c1c5/crypt/zone"), full_name: "oxp_453f7245-5f70-4fe8-b37e-054e56d5c1c5/crypt/zone" } file = sled-storage/src/manager.rs:1331
5532024-11-18T18:36:23.290ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5542024-11-18T18:36:23.290ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/5840d858-3782-45e1-a6e0-d60d070ad2c3/crypt/debug"), full_name: "oxp_5840d858-3782-45e1-a6e0-d60d070ad2c3/crypt/debug" } file = sled-storage/src/manager.rs:1331
5552024-11-18T18:36:23.372ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5562024-11-18T18:36:23.372ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/4be98a0c-ab4c-4309-948b-04d776603876/crypt/zone/oxz_clickhouse_55ab4bd1-3e97-453c-b90c-176360dea573"), full_name: "oxp_4be98a0c-ab4c-4309-948b-04d776603876/crypt/zone/oxz_clickhouse_55ab4bd1-3e97-453c-b90c-176360dea573" } file = sled-storage/src/manager.rs:1331
5572024-11-18T18:36:23.491ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5582024-11-18T18:36:23.491ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/4be98a0c-ab4c-4309-948b-04d776603876/crypt/debug"), full_name: "oxp_4be98a0c-ab4c-4309-948b-04d776603876/crypt/debug" } file = sled-storage/src/manager.rs:1331
5592024-11-18T18:36:23.578ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5602024-11-18T18:36:23.578ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/4be98a0c-ab4c-4309-948b-04d776603876/crypt/zone/oxz_internal_dns_55213fec-a100-46b7-9721-e06d9201c8d5"), full_name: "oxp_4be98a0c-ab4c-4309-948b-04d776603876/crypt/zone/oxz_internal_dns_55213fec-a100-46b7-9721-e06d9201c8d5" } file = sled-storage/src/manager.rs:1331
5612024-11-18T18:36:23.698ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5622024-11-18T18:36:23.698ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/0199d4fd-4090-4a7c-83e6-040814eb79bb/crypt/zone/oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60"), full_name: "oxp_0199d4fd-4090-4a7c-83e6-040814eb79bb/crypt/zone/oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60" } file = sled-storage/src/manager.rs:1331
5632024-11-18T18:36:23.751ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 17491455679030955142 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
5642024-11-18T18:36:23.751ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 9783823515617547413 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
5652024-11-18T18:36:23.817ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5662024-11-18T18:36:23.817ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/131b4d69-dbb0-4397-9284-176e4a3c973d/crypt/zone/oxz_crucible_3aad33dc-2f67-4a47-b973-90de986a5f65"), full_name: "oxp_131b4d69-dbb0-4397-9284-176e4a3c973d/crypt/zone/oxz_crucible_3aad33dc-2f67-4a47-b973-90de986a5f65" } file = sled-storage/src/manager.rs:1331
5672024-11-18T18:36:23.936ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5682024-11-18T18:36:23.936ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/131b4d69-dbb0-4397-9284-176e4a3c973d/crypt/zone"), full_name: "oxp_131b4d69-dbb0-4397-9284-176e4a3c973d/crypt/zone" } file = sled-storage/src/manager.rs:1331
5692024-11-18T18:36:23.969ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::2 exists - OK file = sled-agent/src/services.rs:4293
5702024-11-18T18:36:23.969ZINFOSledAgent (ServiceManager): Ensuring there is a default route file = sled-agent/src/services.rs:4303 gateway = fd00:1122:3344:101::1
5712024-11-18T18:36:23.976ZINFOSledAgent (ServiceManager): Default route already exists file = sled-agent/src/services.rs:4317 gateway = fd00:1122:3344:101::1
5722024-11-18T18:36:23.976ZINFOSledAgent (ServiceManager): configuring dendrite service file = sled-agent/src/services.rs:4382
5732024-11-18T18:36:24.028ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5742024-11-18T18:36:24.028ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/0199d4fd-4090-4a7c-83e6-040814eb79bb/crypt/debug"), full_name: "oxp_0199d4fd-4090-4a7c-83e6-040814eb79bb/crypt/debug" } file = sled-storage/src/manager.rs:1331
5752024-11-18T18:36:24.119ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5762024-11-18T18:36:24.120ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/5840d858-3782-45e1-a6e0-d60d070ad2c3/crypt/zone/oxz_crucible_e782f5fc-076e-414c-b87e-6b09d1bdfaeb"), full_name: "oxp_5840d858-3782-45e1-a6e0-d60d070ad2c3/crypt/zone/oxz_crucible_e782f5fc-076e-414c-b87e-6b09d1bdfaeb" } file = sled-storage/src/manager.rs:1331
5772024-11-18T18:36:24.127ZINFOSledAgent (ServiceManager): refreshed dendrite service with new configuration file = sled-agent/src/services.rs:4448
5782024-11-18T18:36:24.127ZINFOSledAgent (ServiceManager): configuring lldp service file = sled-agent/src/services.rs:4475
5792024-11-18T18:36:24.166ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/ee2b3f44-b73f-4358-9df7-edb19f5079e2) file = sled-agent/src/probe_manager.rs:231 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
5802024-11-18T18:36:24.170ZINFOSledAgent (ServiceManager): refreshed lldpd service with new configuration file = sled-agent/src/services.rs:4488
5812024-11-18T18:36:24.170ZINFOSledAgent (ServiceManager): configuring MGS service file = sled-agent/src/services.rs:4334
5822024-11-18T18:36:24.221ZINFOSledAgent (ServiceManager): refreshed MGS service with new configuration file = sled-agent/src/services.rs:4376
5832024-11-18T18:36:24.221ZINFOSledAgent (ServiceManager): configuring wicketd file = sled-agent/src/services.rs:4455 rack_subnet = fd00:1122:3344::
5842024-11-18T18:36:24.242ZINFOSledAgent (ServiceManager): refreshed wicketd service with new configuration file = sled-agent/src/services.rs:4466
5852024-11-18T18:36:24.242ZINFOSledAgent (ServiceManager): configuring mgd service file = sled-agent/src/services.rs:4508
5862024-11-18T18:36:24.252ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5872024-11-18T18:36:24.254ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/4f35fd7e-936a-4e09-946b-7cb174580897/crypt/zone/oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a"), full_name: "oxp_4f35fd7e-936a-4e09-946b-7cb174580897/crypt/zone/oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a" } file = sled-storage/src/manager.rs:1331
5882024-11-18T18:36:24.309ZINFOSledAgent (ServiceManager): refreshed mgd service with new configuration file = sled-agent/src/services.rs:4540
5892024-11-18T18:36:24.309ZINFOSledAgent (ServiceManager): configuring mg-ddm service file = sled-agent/src/services.rs:4546
5902024-11-18T18:36:24.384ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5912024-11-18T18:36:24.384ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/b09b6e80-5341-4ded-a520-3f312403e198/crypt/zone/oxz_crucible_2735d4d6-073c-4860-ae50-e3742cc2cefa"), full_name: "oxp_b09b6e80-5341-4ded-a520-3f312403e198/crypt/zone/oxz_crucible_2735d4d6-073c-4860-ae50-e3742cc2cefa" } file = sled-storage/src/manager.rs:1331
5922024-11-18T18:36:24.408ZINFOSledAgent (ServiceManager): refreshed mg-ddm service with new configuration file = sled-agent/src/services.rs:4587
5932024-11-18T18:36:24.408ZINFOSledAgent (ServiceManager): Determining physical location of our switch zone at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:370
5942024-11-18T18:36:24.448ZINFOSledAgent (ServiceManager): Initializing 1 Uplinks on Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:418
5952024-11-18T18:36:24.452ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/ee2b3f44-b73f-4358-9df7-edb19f5079e2) file = sled-agent/src/nexus.rs:527 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
5962024-11-18T18:36:24.490ZINFOSledAgent (ServiceManager): Building Port Configuration file = sled-agent/src/bootstrap/early_networking.rs:686
5972024-11-18T18:36:24.490ZINFOSledAgent (ServiceManager): Checking dendrite uptime file = sled-agent/src/bootstrap/early_networking.rs:731
5982024-11-18T18:36:24.491ZINFOSledAgent (ServiceManager): Dendrite online file = sled-agent/src/bootstrap/early_networking.rs:734 uptime = 1
5992024-11-18T18:36:24.491ZINFOSledAgent (ServiceManager): Configuring default uplink on switch config = PortSettings {\n links: {\n "0": LinkSettings {\n addrs: [\n 10.151.2.174,\n ],\n params: LinkCreate {\n autoneg: false,\n fec: Some(\n None,\n ),\n kr: false,\n lane: Some(\n LinkId(\n 0,\n ),\n ),\n speed: Speed40G,\n tx_eq: None,\n },\n },\n },\n} file = sled-agent/src/bootstrap/early_networking.rs:440
6002024-11-18T18:36:24.508ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6012024-11-18T18:36:24.508ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/0199d4fd-4090-4a7c-83e6-040814eb79bb/crypt/zone/oxz_crucible_pantry_b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46"), full_name: "oxp_0199d4fd-4090-4a7c-83e6-040814eb79bb/crypt/zone/oxz_crucible_pantry_b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46" } file = sled-storage/src/manager.rs:1331
6022024-11-18T18:36:24.539ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
6032024-11-18T18:36:24.572ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 22.646570327s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
6042024-11-18T18:36:24.635ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6052024-11-18T18:36:24.635ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/df4faf1b-f747-4cda-8481-deaa8523a533/crypt/zone/oxz_crucible_6062ae2a-331c-4cb6-a9b5-eb4310b33530"), full_name: "oxp_df4faf1b-f747-4cda-8481-deaa8523a533/crypt/zone/oxz_crucible_6062ae2a-331c-4cb6-a9b5-eb4310b33530" } file = sled-storage/src/manager.rs:1331
6062024-11-18T18:36:24.754ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6072024-11-18T18:36:24.754ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/453f7245-5f70-4fe8-b37e-054e56d5c1c5/crypt/zone/oxz_crucible_81488621-5898-41d4-a112-d3e574fb2d33"), full_name: "oxp_453f7245-5f70-4fe8-b37e-054e56d5c1c5/crypt/zone/oxz_crucible_81488621-5898-41d4-a112-d3e574fb2d33" } file = sled-storage/src/manager.rs:1331
6082024-11-18T18:36:24.875ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6092024-11-18T18:36:24.875ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_4f35fd7e-936a-4e09-946b-7cb174580897/crypt/cockroachdb" } file = sled-storage/src/manager.rs:1331
6102024-11-18T18:36:24.986ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6112024-11-18T18:36:24.986ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/f23dc95a-d65a-4537-b65e-161954e059fa/crypt/zone/oxz_crucible_10904f20-d090-41dc-9a54-dab632f01d4a"), full_name: "oxp_f23dc95a-d65a-4537-b65e-161954e059fa/crypt/zone/oxz_crucible_10904f20-d090-41dc-9a54-dab632f01d4a" } file = sled-storage/src/manager.rs:1331
6122024-11-18T18:36:25.100ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6132024-11-18T18:36:25.100ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/0199d4fd-4090-4a7c-83e6-040814eb79bb/crypt/zone"), full_name: "oxp_0199d4fd-4090-4a7c-83e6-040814eb79bb/crypt/zone" } file = sled-storage/src/manager.rs:1331
6142024-11-18T18:36:25.170ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 12802533969288132606 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
6152024-11-18T18:36:25.171ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 9234514238289780840 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
6162024-11-18T18:36:25.171ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/ee2b3f44-b73f-4358-9df7-edb19f5079e2) file = sled-agent/src/probe_manager.rs:231 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
6172024-11-18T18:36:25.181ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6182024-11-18T18:36:25.181ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_4be98a0c-ab4c-4309-948b-04d776603876/crypt/cockroachdb" } file = sled-storage/src/manager.rs:1331
6192024-11-18T18:36:25.293ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6202024-11-18T18:36:25.293ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/4f35fd7e-936a-4e09-946b-7cb174580897/crypt/zone/oxz_nexus_88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d"), full_name: "oxp_4f35fd7e-936a-4e09-946b-7cb174580897/crypt/zone/oxz_nexus_88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d" } file = sled-storage/src/manager.rs:1331
6212024-11-18T18:36:25.411ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6222024-11-18T18:36:25.411ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/131b4d69-dbb0-4397-9284-176e4a3c973d/crypt/zone/oxz_crucible_pantry_1fa25dd7-d8f9-49ef-bb20-c6bad413aace"), full_name: "oxp_131b4d69-dbb0-4397-9284-176e4a3c973d/crypt/zone/oxz_crucible_pantry_1fa25dd7-d8f9-49ef-bb20-c6bad413aace" } file = sled-storage/src/manager.rs:1331
6232024-11-18T18:36:25.530ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6242024-11-18T18:36:25.530ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_131b4d69-dbb0-4397-9284-176e4a3c973d/crucible" } file = sled-storage/src/manager.rs:1331
6252024-11-18T18:36:25.636ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6262024-11-18T18:36:25.636ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/df4faf1b-f747-4cda-8481-deaa8523a533/crypt/zone"), full_name: "oxp_df4faf1b-f747-4cda-8481-deaa8523a533/crypt/zone" } file = sled-storage/src/manager.rs:1331
6272024-11-18T18:36:25.717ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6282024-11-18T18:36:25.717ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_df4faf1b-f747-4cda-8481-deaa8523a533/crucible" } file = sled-storage/src/manager.rs:1331
6292024-11-18T18:36:25.823ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/9216f0f9-29c9-4d33-8c71-0f237e1331a0/config/.omicron-datasets.json.tmp file = common/src/ledger.rs:199 request = datasets_ensure
6302024-11-18T18:36:25.824ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/c23ca4a6-dcd8-4835-a278-26664c492a4d/config/.omicron-datasets.json.tmp file = common/src/ledger.rs:199 request = datasets_ensure
6312024-11-18T18:36:25.824ZINFOSledAgent (StorageManager): Received DetectedRawDisksChanged { raw_disks: {}, tx: ... } file = sled-storage/src/manager.rs:603
6322024-11-18T18:36:25.824ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
6332024-11-18T18:36:25.824ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
6342024-11-18T18:36:25.824ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
6352024-11-18T18:36:25.824ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
6362024-11-18T18:36:25.824ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
6372024-11-18T18:36:25.824ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
6382024-11-18T18:36:25.824ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
6392024-11-18T18:36:25.824ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
6402024-11-18T18:36:25.824ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
6412024-11-18T18:36:25.824ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
6422024-11-18T18:36:25.824ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
6432024-11-18T18:36:25.824ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
6442024-11-18T18:36:25.824ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
6452024-11-18T18:36:25.824ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
6462024-11-18T18:36:25.824ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
6472024-11-18T18:36:25.824ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
6482024-11-18T18:36:25.824ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
6492024-11-18T18:36:25.824ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
6502024-11-18T18:36:25.824ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
6512024-11-18T18:36:25.824ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
6522024-11-18T18:36:25.824ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
6532024-11-18T18:36:25.824ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
6542024-11-18T18:36:25.824ZINFOSledAgent: datasets ensure: Updated storage file = sled-agent/src/sled_agent.rs:852 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
6552024-11-18T18:36:25.824ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 6952611 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:56563 req_id = 625634da-847c-4dfd-98e3-3027ee1406a0 response_code = 200 uri = /datasets
6562024-11-18T18:36:25.856ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(2), zones: [OmicronZoneConfig { id: 55213fec-a100-46b7-9721-e06d9201c8d5 (service), filesystem_pool: Some(ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 07f4fc36-9902-4aeb-a7d9-f99de01ff5c2 (service), filesystem_pool: Some(ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: b160705f-2e55-4b76-ba79-187b4901f005 (service), filesystem_pool: Some(ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }] } file = sled-agent/src/rack_setup/service.rs:538 sled_address = [fd00:1122:3344:101::1]:12345
6572024-11-18T18:36:25.856ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:37966
6582024-11-18T18:36:25.856ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 55213fec-a100-46b7-9721-e06d9201c8d5, dataset_name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
6592024-11-18T18:36:25.856ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 55213fec-a100-46b7-9721-e06d9201c8d5, dataset_name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: InternalDns }, responder: ... }
6602024-11-18T18:36:25.891ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 07f4fc36-9902-4aeb-a7d9-f99de01ff5c2, dataset_name: DatasetName { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
6612024-11-18T18:36:25.891ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 07f4fc36-9902-4aeb-a7d9-f99de01ff5c2, dataset_name: DatasetName { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, kind: InternalDns }, responder: ... }
6622024-11-18T18:36:25.925ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: b160705f-2e55-4b76-ba79-187b4901f005, dataset_name: DatasetName { pool_name: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
6632024-11-18T18:36:25.925ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: b160705f-2e55-4b76-ba79-187b4901f005, dataset_name: DatasetName { pool_name: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }, kind: InternalDns }, responder: ... }
6642024-11-18T18:36:25.961ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
6652024-11-18T18:36:25.961ZINFOSledAgent (ServiceManager): No ledger in /pool/int/9216f0f9-29c9-4d33-8c71-0f237e1331a0/config/omicron-zones.json file = common/src/ledger.rs:188
6662024-11-18T18:36:25.961ZINFOSledAgent (ServiceManager): No ledger in /pool/int/c23ca4a6-dcd8-4835-a278-26664c492a4d/config/omicron-zones.json file = common/src/ledger.rs:188
6672024-11-18T18:36:25.961ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
6682024-11-18T18:36:26.012ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
6692024-11-18T18:36:26.012ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
6702024-11-18T18:36:26.012ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
6712024-11-18T18:36:26.068ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_55213fec-a100-46b7-9721-e06d9201c8d5 file = illumos-utils/src/zone.rs:325
6722024-11-18T18:36:26.072ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_07f4fc36-9902-4aeb-a7d9-f99de01ff5c2 file = illumos-utils/src/zone.rs:325
6732024-11-18T18:36:26.077ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_b160705f-2e55-4b76-ba79-187b4901f005 file = illumos-utils/src/zone.rs:325
6742024-11-18T18:36:26.089ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_55213fec-a100-46b7-9721-e06d9201c8d5 file = illumos-utils/src/zone.rs:363
6752024-11-18T18:36:26.094ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_07f4fc36-9902-4aeb-a7d9-f99de01ff5c2 file = illumos-utils/src/zone.rs:363
6762024-11-18T18:36:26.099ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_b160705f-2e55-4b76-ba79-187b4901f005 file = illumos-utils/src/zone.rs:363
6772024-11-18T18:36:26.173ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/ee2b3f44-b73f-4358-9df7-edb19f5079e2) file = sled-agent/src/probe_manager.rs:231 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
6782024-11-18T18:36:26.449ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/ee2b3f44-b73f-4358-9df7-edb19f5079e2) file = sled-agent/src/nexus.rs:527 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
6792024-11-18T18:36:27.176ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/ee2b3f44-b73f-4358-9df7-edb19f5079e2) file = sled-agent/src/probe_manager.rs:231 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
6802024-11-18T18:36:27.715ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_b160705f-2e55-4b76-ba79-187b4901f005: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService3'/> <propval type="astring" name="gateway" value='fd00:1122:3344:3::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:3::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:3::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:3::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
6812024-11-18T18:36:27.715ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:3::, width: 64 }
6822024-11-18T18:36:28.195ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_55213fec-a100-46b7-9721-e06d9201c8d5: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService1'/> <propval type="astring" name="gateway" value='fd00:1122:3344:1::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:1::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:1::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:1::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
6832024-11-18T18:36:28.673ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_07f4fc36-9902-4aeb-a7d9-f99de01ff5c2: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService2'/> <propval type="astring" name="gateway" value='fd00:1122:3344:2::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:2::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:2::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:2::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
6842024-11-18T18:36:28.673ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:1::, width: 64 }
6852024-11-18T18:36:28.673ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_b160705f-2e55-4b76-ba79-187b4901f005 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_b160705f-2e55-4b76-ba79-187b4901f005
6862024-11-18T18:36:28.677ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:2::, width: 64 }
6872024-11-18T18:36:28.677ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_55213fec-a100-46b7-9721-e06d9201c8d5 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_55213fec-a100-46b7-9721-e06d9201c8d5
6882024-11-18T18:36:28.682ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_07f4fc36-9902-4aeb-a7d9-f99de01ff5c2 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_07f4fc36-9902-4aeb-a7d9-f99de01ff5c2
6892024-11-18T18:36:28.809ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_b160705f-2e55-4b76-ba79-187b4901f005") failed: Property not found. retry in 29.12122ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_b160705f-2e55-4b76-ba79-187b4901f005
6902024-11-18T18:36:28.816ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_55213fec-a100-46b7-9721-e06d9201c8d5") failed: Property not found. retry in 26.376508ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_55213fec-a100-46b7-9721-e06d9201c8d5
6912024-11-18T18:36:28.823ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_07f4fc36-9902-4aeb-a7d9-f99de01ff5c2") failed: Property not found. retry in 49.658777ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_07f4fc36-9902-4aeb-a7d9-f99de01ff5c2
6922024-11-18T18:36:28.846ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_b160705f-2e55-4b76-ba79-187b4901f005") failed: Property not found. retry in 98.985882ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_b160705f-2e55-4b76-ba79-187b4901f005
6932024-11-18T18:36:28.853ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_55213fec-a100-46b7-9721-e06d9201c8d5") failed: Property not found. retry in 135.313899ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_55213fec-a100-46b7-9721-e06d9201c8d5
6942024-11-18T18:36:28.956ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_07f4fc36-9902-4aeb-a7d9-f99de01ff5c2") failed: Property not found. retry in 110.961483ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_07f4fc36-9902-4aeb-a7d9-f99de01ff5c2
6952024-11-18T18:36:29.026ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_b160705f-2e55-4b76-ba79-187b4901f005") failed: Property not found. retry in 200.870858ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_b160705f-2e55-4b76-ba79-187b4901f005
6962024-11-18T18:36:29.097ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_55213fec-a100-46b7-9721-e06d9201c8d5") failed: Property not found. retry in 150.097809ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_55213fec-a100-46b7-9721-e06d9201c8d5
6972024-11-18T18:36:29.168ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_07f4fc36-9902-4aeb-a7d9-f99de01ff5c2") failed: Property not found. retry in 296.442913ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_07f4fc36-9902-4aeb-a7d9-f99de01ff5c2
6982024-11-18T18:36:29.371ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_b160705f-2e55-4b76-ba79-187b4901f005") failed: Property not found. retry in 287.712339ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_b160705f-2e55-4b76-ba79-187b4901f005
6992024-11-18T18:36:29.423ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_55213fec-a100-46b7-9721-e06d9201c8d5") failed: Property not found. retry in 373.072305ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_55213fec-a100-46b7-9721-e06d9201c8d5
7002024-11-18T18:36:29.506ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_07f4fc36-9902-4aeb-a7d9-f99de01ff5c2") failed: Property not found. retry in 317.906285ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_07f4fc36-9902-4aeb-a7d9-f99de01ff5c2
7012024-11-18T18:36:29.670ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_b160705f-2e55-4b76-ba79-187b4901f005") failed: Property not found. retry in 690.483262ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_b160705f-2e55-4b76-ba79-187b4901f005
7022024-11-18T18:36:29.806ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_55213fec-a100-46b7-9721-e06d9201c8d5") failed: Property not found. retry in 880.081942ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_55213fec-a100-46b7-9721-e06d9201c8d5
7032024-11-18T18:36:29.834ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_07f4fc36-9902-4aeb-a7d9-f99de01ff5c2") failed: Property not found. retry in 747.480574ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_07f4fc36-9902-4aeb-a7d9-f99de01ff5c2
7042024-11-18T18:36:30.372ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_b160705f-2e55-4b76-ba79-187b4901f005") failed: Property not found. retry in 1.344933992s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_b160705f-2e55-4b76-ba79-187b4901f005
7052024-11-18T18:36:30.591ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_07f4fc36-9902-4aeb-a7d9-f99de01ff5c2") failed: Property not found. retry in 659.715076ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_07f4fc36-9902-4aeb-a7d9-f99de01ff5c2
7062024-11-18T18:36:30.696ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_55213fec-a100-46b7-9721-e06d9201c8d5") failed: Property not found. retry in 1.428566459s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_55213fec-a100-46b7-9721-e06d9201c8d5
7072024-11-18T18:36:31.263ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_07f4fc36-9902-4aeb-a7d9-f99de01ff5c2") failed: Property not found. retry in 1.090282072s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_07f4fc36-9902-4aeb-a7d9-f99de01ff5c2
7082024-11-18T18:36:31.726ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_b160705f-2e55-4b76-ba79-187b4901f005") failed: Property not found. retry in 1.372257705s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_b160705f-2e55-4b76-ba79-187b4901f005
7092024-11-18T18:36:32.134ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_55213fec-a100-46b7-9721-e06d9201c8d5") failed: Property not found. retry in 1.416097583s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_55213fec-a100-46b7-9721-e06d9201c8d5
7102024-11-18T18:36:32.361ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_07f4fc36-9902-4aeb-a7d9-f99de01ff5c2") failed: Property not found. retry in 1.292966156s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_07f4fc36-9902-4aeb-a7d9-f99de01ff5c2
7112024-11-18T18:36:33.108ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_b160705f-2e55-4b76-ba79-187b4901f005") failed: Property not found. retry in 657.226993ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_b160705f-2e55-4b76-ba79-187b4901f005
7122024-11-18T18:36:33.560ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_55213fec-a100-46b7-9721-e06d9201c8d5") failed: Property not found. retry in 766.570111ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_55213fec-a100-46b7-9721-e06d9201c8d5
7132024-11-18T18:36:33.664ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_07f4fc36-9902-4aeb-a7d9-f99de01ff5c2") failed: Property not found. retry in 837.387198ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_07f4fc36-9902-4aeb-a7d9-f99de01ff5c2
7142024-11-18T18:36:33.753ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 17491455679030955142 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
7152024-11-18T18:36:33.753ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 9783823515617547413 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
7162024-11-18T18:36:33.775ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_b160705f-2e55-4b76-ba79-187b4901f005") failed: Property not found. retry in 1.237814953s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_b160705f-2e55-4b76-ba79-187b4901f005
7172024-11-18T18:36:35.032ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3311 zone = oxz_internal_dns_55213fec-a100-46b7-9721-e06d9201c8d5
7182024-11-18T18:36:35.032ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3311 zone = oxz_internal_dns_07f4fc36-9902-4aeb-a7d9-f99de01ff5c2
7192024-11-18T18:36:35.033ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3311 zone = oxz_internal_dns_b160705f-2e55-4b76-ba79-187b4901f005
7202024-11-18T18:36:35.033ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/9216f0f9-29c9-4d33-8c71-0f237e1331a0/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
7212024-11-18T18:36:35.033ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/c23ca4a6-dcd8-4835-a278-26664c492a4d/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
7222024-11-18T18:36:35.033ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 9176962 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:37966 req_id = 53a2af84-e232-43a2-ae7b-91ea251de7cc response_code = 204 uri = /omicron-zones
7232024-11-18T18:36:35.035ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:1::1]:5353 file = sled-agent/src/rack_setup/service.rs:655
7242024-11-18T18:36:35.172ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 12802533969288132606 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
7252024-11-18T18:36:35.173ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 9234514238289780840 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
7262024-11-18T18:36:35.437ZWARNSledAgent (RSS): failed to write DNS configuration (will retry in 114.830546ms) dns_config_addr = [fd00:1122:3344:1::1]:5353 error_message = Communication Error: error sending request for url (http://[fd00:1122:3344:1::1]:5353/config) file = sled-agent/src/rack_setup/service.rs:675
7272024-11-18T18:36:35.570ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:2::1]:5353 file = sled-agent/src/rack_setup/service.rs:655
7282024-11-18T18:36:35.619ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:3::1]:5353 file = sled-agent/src/rack_setup/service.rs:655
7292024-11-18T18:36:35.668ZINFOSledAgent (RSS): Configured all DNS servers file = sled-agent/src/rack_setup/service.rs:691
7302024-11-18T18:36:35.668ZINFOSledAgent (RSS): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
7312024-11-18T18:36:35.701ZINFOSledAgent (RSS): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
7322024-11-18T18:36:35.702ZINFOSledAgent (RSS): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
7332024-11-18T18:36:35.702ZINFOSledAgent (RSS): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
7342024-11-18T18:36:35.733ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(3), zones: [OmicronZoneConfig { id: 55213fec-a100-46b7-9721-e06d9201c8d5 (service), filesystem_pool: Some(ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 07f4fc36-9902-4aeb-a7d9-f99de01ff5c2 (service), filesystem_pool: Some(ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: b160705f-2e55-4b76-ba79-187b4901f005 (service), filesystem_pool: Some(ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 6dfcb5a1-e0b1-44a6-97dc-3d4ea97e4f50 (service), filesystem_pool: Some(ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: c9bbb6b8-8989-4090-b169-2e9a21574d6c, kind: Service { id: 6dfcb5a1-e0b1-44a6-97dc-3d4ea97e4f50 }, name: Name("ntp-6dfcb5a1-e0b1-44a6-97dc-3d4ea97e4f50"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 148, 216])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }] } file = sled-agent/src/rack_setup/service.rs:538 sled_address = [fd00:1122:3344:101::1]:12345
7352024-11-18T18:36:35.733ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:63917
7362024-11-18T18:36:35.733ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 55213fec-a100-46b7-9721-e06d9201c8d5, dataset_name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
7372024-11-18T18:36:35.733ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 55213fec-a100-46b7-9721-e06d9201c8d5, dataset_name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: InternalDns }, responder: ... }
7382024-11-18T18:36:35.774ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 07f4fc36-9902-4aeb-a7d9-f99de01ff5c2, dataset_name: DatasetName { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
7392024-11-18T18:36:35.774ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 07f4fc36-9902-4aeb-a7d9-f99de01ff5c2, dataset_name: DatasetName { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, kind: InternalDns }, responder: ... }
7402024-11-18T18:36:35.810ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: b160705f-2e55-4b76-ba79-187b4901f005, dataset_name: DatasetName { pool_name: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
7412024-11-18T18:36:35.810ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: b160705f-2e55-4b76-ba79-187b4901f005, dataset_name: DatasetName { pool_name: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }, kind: InternalDns }, responder: ... }
7422024-11-18T18:36:35.846ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
7432024-11-18T18:36:35.846ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/9216f0f9-29c9-4d33-8c71-0f237e1331a0/config/omicron-zones.json file = common/src/ledger.rs:177
7442024-11-18T18:36:35.846ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/c23ca4a6-dcd8-4835-a278-26664c492a4d/config/omicron-zones.json file = common/src/ledger.rs:177
7452024-11-18T18:36:35.846ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
7462024-11-18T18:36:35.857ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
7472024-11-18T18:36:35.889ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
7482024-11-18T18:36:35.890ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
7492024-11-18T18:36:35.890ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
7502024-11-18T18:36:35.919ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:430 port = Port { inner: PortInner(PortData { name: "opte0", ip: 172.30.3.5, mac: MacAddr6([168, 64, 37, 255, 148, 216]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.3.1 } }) }
7512024-11-18T18:36:35.919ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1264 zone_type = boundary_ntp
7522024-11-18T18:36:35.920ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.169 file = clients/dpd-client/src/lib.rs:83
7532024-11-18T18:36:35.920ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.169 file = clients/dpd-client/src/lib.rs:90
7542024-11-18T18:36:35.920ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.169 file = clients/dpd-client/src/lib.rs:118
7552024-11-18T18:36:35.921ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
7562024-11-18T18:36:35.945ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_ntp_6dfcb5a1-e0b1-44a6-97dc-3d4ea97e4f50 file = illumos-utils/src/zone.rs:325
7572024-11-18T18:36:35.964ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_ntp_6dfcb5a1-e0b1-44a6-97dc-3d4ea97e4f50 file = illumos-utils/src/zone.rs:363
7582024-11-18T18:36:36.575ZINFOSledAgent (ServiceManager): Profile for oxz_ntp_6dfcb5a1-e0b1-44a6-97dc-3d4ea97e4f50: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService4'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::1b'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/chrony-setup"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="allow" value='fd00:1122:3344:100::/56'/> <propval type="boolean" name="boundary" value='true'/> <propval type="astring" name="boundary_pool" value='boundary-ntp.control-plane.oxide.internal'/> <propval type="astring" name="server" value='0.pool.ntp.org'/> </property_group> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="1.1.1.1" /> <value_node value="9.9.9.9" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/ntp"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte0'/> <propval type="astring" name="gateway" value='172.30.3.1'/> <propval type="astring" name="ip" value='172.30.3.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
7592024-11-18T18:36:36.575ZINFOSledAgent (ServiceManager): Booting oxz_ntp_6dfcb5a1-e0b1-44a6-97dc-3d4ea97e4f50 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_ntp_6dfcb5a1-e0b1-44a6-97dc-3d4ea97e4f50
7602024-11-18T18:36:36.713ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_6dfcb5a1-e0b1-44a6-97dc-3d4ea97e4f50") failed: Property not found. retry in 64.439196ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_6dfcb5a1-e0b1-44a6-97dc-3d4ea97e4f50
7612024-11-18T18:36:36.790ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_6dfcb5a1-e0b1-44a6-97dc-3d4ea97e4f50") failed: Property not found. retry in 105.331398ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_6dfcb5a1-e0b1-44a6-97dc-3d4ea97e4f50
7622024-11-18T18:36:36.913ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_6dfcb5a1-e0b1-44a6-97dc-3d4ea97e4f50") failed: Property not found. retry in 252.600019ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_6dfcb5a1-e0b1-44a6-97dc-3d4ea97e4f50
7632024-11-18T18:36:37.180ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_6dfcb5a1-e0b1-44a6-97dc-3d4ea97e4f50") failed: Property not found. retry in 552.079084ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_6dfcb5a1-e0b1-44a6-97dc-3d4ea97e4f50
7642024-11-18T18:36:37.741ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_6dfcb5a1-e0b1-44a6-97dc-3d4ea97e4f50") failed: Property not found. retry in 912.764803ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_6dfcb5a1-e0b1-44a6-97dc-3d4ea97e4f50
7652024-11-18T18:36:38.663ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_6dfcb5a1-e0b1-44a6-97dc-3d4ea97e4f50") failed: Property not found. retry in 1.264125816s file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_6dfcb5a1-e0b1-44a6-97dc-3d4ea97e4f50
7662024-11-18T18:36:39.937ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_6dfcb5a1-e0b1-44a6-97dc-3d4ea97e4f50") failed: Property not found. retry in 1.355693071s file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_6dfcb5a1-e0b1-44a6-97dc-3d4ea97e4f50
7672024-11-18T18:36:41.302ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_6dfcb5a1-e0b1-44a6-97dc-3d4ea97e4f50") failed: Property not found. retry in 1.042399412s file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_6dfcb5a1-e0b1-44a6-97dc-3d4ea97e4f50
7682024-11-18T18:36:42.365ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3311 zone = oxz_ntp_6dfcb5a1-e0b1-44a6-97dc-3d4ea97e4f50
7692024-11-18T18:36:42.365ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/9216f0f9-29c9-4d33-8c71-0f237e1331a0/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
7702024-11-18T18:36:42.366ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/c23ca4a6-dcd8-4835-a278-26664c492a4d/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
7712024-11-18T18:36:42.366ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 6633154 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:63917 req_id = 438d06c8-22b0-4ed9-880f-8dd200317d03 response_code = 204 uri = /omicron-zones
7722024-11-18T18:36:42.368ZINFOSledAgent (RSS): Waiting for rack time synchronization file = sled-agent/src/rack_setup/service.rs:732
7732024-11-18T18:36:42.399ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
7742024-11-18T18:36:42.399ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:40530
7752024-11-18T18:36:42.408ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_6dfcb5a1-e0b1-44a6-97dc-3d4ea97e4f50': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3901
7762024-11-18T18:36:42.408ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 9361 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:40530 req_id = 870e6046-1047-4dd6-9130-8ca42a9d5421 response_code = 500 uri = /timesync
7772024-11-18T18:36:42.411ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
7782024-11-18T18:36:42.547ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
7792024-11-18T18:36:42.548ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:49096
7802024-11-18T18:36:43.756ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 17491455679030955142 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
7812024-11-18T18:36:43.756ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 9783823515617547413 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
7822024-11-18T18:36:44.350ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 4297017578838306490 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
7832024-11-18T18:36:44.528ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 8291995201846061523 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
7842024-11-18T18:36:45.036ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 13979037378220380202 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
7852024-11-18T18:36:45.174ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 12802533969288132606 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
7862024-11-18T18:36:45.174ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 9234514238289780840 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
7872024-11-18T18:36:46.556ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_6dfcb5a1-e0b1-44a6-97dc-3d4ea97e4f50': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3901
7882024-11-18T18:36:46.556ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 4008266 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:49096 req_id = 8dd125ca-8854-4bdc-8fbb-5a7686c29b7c response_code = 500 uri = /timesync
7892024-11-18T18:36:46.558ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
7902024-11-18T18:36:46.679ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
7912024-11-18T18:36:46.679ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:48784
7922024-11-18T18:36:46.688ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_6dfcb5a1-e0b1-44a6-97dc-3d4ea97e4f50': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3901
7932024-11-18T18:36:46.688ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 8893 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:48784 req_id = f49a3411-00d3-4718-84f3-e332d296a44d response_code = 500 uri = /timesync
7942024-11-18T18:36:46.690ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
7952024-11-18T18:36:46.939ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
7962024-11-18T18:36:46.939ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:33544
7972024-11-18T18:36:46.948ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_6dfcb5a1-e0b1-44a6-97dc-3d4ea97e4f50': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3901
7982024-11-18T18:36:46.948ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 8752 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:33544 req_id = b54cc288-7237-4912-8e56-ba9f8439d572 response_code = 500 uri = /timesync
7992024-11-18T18:36:46.949ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
8002024-11-18T18:36:47.233ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
8012024-11-18T18:36:47.233ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:46864
8022024-11-18T18:36:47.243ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 9235 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:46864 req_id = 2fdcdd44-3760-4921-a5e2-2dc13fce6291 response_code = 200 uri = /timesync
8032024-11-18T18:36:47.245ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:740
8042024-11-18T18:36:47.245ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
8052024-11-18T18:36:47.412ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
8062024-11-18T18:36:47.412ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:35192
8072024-11-18T18:36:47.421ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8942 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:35192 req_id = adae0961-2b22-420a-8401-9b1757e6868e response_code = 200 uri = /timesync
8082024-11-18T18:36:47.423ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:740
8092024-11-18T18:36:47.423ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
8102024-11-18T18:36:47.648ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
8112024-11-18T18:36:47.648ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:33883
8122024-11-18T18:36:47.657ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8865 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:33883 req_id = 5c29fe4b-c0b1-4cbb-a309-45e6516f1d66 response_code = 200 uri = /timesync
8132024-11-18T18:36:47.661ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:740
8142024-11-18T18:36:47.661ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
8152024-11-18T18:36:47.973ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
8162024-11-18T18:36:47.973ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:42248
8172024-11-18T18:36:47.981ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8312 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:42248 req_id = 480f88b6-9758-48f2-8f44-941b6292cd24 response_code = 200 uri = /timesync
8182024-11-18T18:36:47.983ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:740
8192024-11-18T18:36:47.983ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
8202024-11-18T18:36:48.247ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
8212024-11-18T18:36:48.247ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:42822
8222024-11-18T18:36:48.256ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8307 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:42822 req_id = 3eee0a19-997c-4e1a-8c5f-d75d1f70d7d3 response_code = 200 uri = /timesync
8232024-11-18T18:36:48.259ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:740
8242024-11-18T18:36:48.259ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
8252024-11-18T18:36:48.657ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
8262024-11-18T18:36:48.657ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:54730
8272024-11-18T18:36:48.666ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8890 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:54730 req_id = d9274c6f-e441-4739-9275-c43131346c2e response_code = 200 uri = /timesync
8282024-11-18T18:36:48.668ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:740
8292024-11-18T18:36:48.668ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
8302024-11-18T18:36:49.319ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
8312024-11-18T18:36:49.319ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:48069
8322024-11-18T18:36:49.328ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8626 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:48069 req_id = c26f7e74-6d2e-4d98-9dae-d134c6defbd5 response_code = 200 uri = /timesync
8332024-11-18T18:36:49.329ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1731955007.1113548, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:740
8342024-11-18T18:36:49.329ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
8352024-11-18T18:36:49.983ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
8362024-11-18T18:36:49.983ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:34487
8372024-11-18T18:36:49.992ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8749 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:34487 req_id = 3b27dcfa-950f-48d6-a755-56f41eba90ed response_code = 200 uri = /timesync
8382024-11-18T18:36:49.996ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1731955007.1113548, correction: -0.426573664 } file = sled-agent/src/rack_setup/service.rs:740
8392024-11-18T18:36:49.996ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
8402024-11-18T18:36:51.058ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
8412024-11-18T18:36:51.058ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:41846
8422024-11-18T18:36:51.066ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7829 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:41846 req_id = 29c6c0cc-c5b3-4f5f-b424-4cb889470698 response_code = 200 uri = /timesync
8432024-11-18T18:36:51.068ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 1753455535, ip_addr: 104.131.155.175, stratum: 3, ref_time: 1731955009.8317497, correction: -0.424282283 } file = sled-agent/src/rack_setup/service.rs:740
8442024-11-18T18:36:51.068ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
8452024-11-18T18:36:51.660ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
8462024-11-18T18:36:51.660ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:50658
8472024-11-18T18:36:51.670ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 9270 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:50658 req_id = e8d09ddc-0918-4423-bbbd-9aeb6431384b response_code = 200 uri = /timesync
8482024-11-18T18:36:51.673ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 1753455535, ip_addr: 104.131.155.175, stratum: 3, ref_time: 1731955010.3490613, correction: -0.422905505 } file = sled-agent/src/rack_setup/service.rs:740
8492024-11-18T18:36:51.673ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
8502024-11-18T18:36:52.598ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 14973738965599029016 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
8512024-11-18T18:36:52.598ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 8425061366609949235 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
8522024-11-18T18:36:53.290ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
8532024-11-18T18:36:53.290ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:37047
8542024-11-18T18:36:53.299ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8456 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:37047 req_id = e2eeb991-5036-4a5f-87bb-1ea58060acda response_code = 200 uri = /timesync
8552024-11-18T18:36:53.301ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 1753455535, ip_addr: 104.131.155.175, stratum: 3, ref_time: 1731955011.7672446, correction: -0.418586046 } file = sled-agent/src/rack_setup/service.rs:740
8562024-11-18T18:36:53.301ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
8572024-11-18T18:36:53.983ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 17491455679030955142 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
8582024-11-18T18:36:53.983ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 9783823515617547413 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
8592024-11-18T18:36:54.079ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
8602024-11-18T18:36:54.079ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:53198
8612024-11-18T18:36:54.089ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 9230 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:53198 req_id = 9f527e50-3dcf-45e7-bed7-3085c110a831 response_code = 200 uri = /timesync
8622024-11-18T18:36:54.092ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 1753455535, ip_addr: 104.131.155.175, stratum: 3, ref_time: 1731955012.5710292, correction: -0.416359693 } file = sled-agent/src/rack_setup/service.rs:740
8632024-11-18T18:36:54.092ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
8642024-11-18T18:36:54.424ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:46678
8652024-11-18T18:36:54.424ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 69 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:46678 req_id = 92bbda71-f70c-4e03-8790-3f85097b7022 response_code = 200 uri = /baseboard
8662024-11-18T18:36:54.577ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 4297017578838306490 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
8672024-11-18T18:36:54.753ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 8291995201846061523 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
8682024-11-18T18:36:55.260ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 13979037378220380202 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
8692024-11-18T18:36:55.370ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
8702024-11-18T18:36:55.370ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:45724
8712024-11-18T18:36:55.379ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8976 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:45724 req_id = 5bb2c0fd-bb2c-46a0-ab96-75a95c534af3 response_code = 200 uri = /timesync
8722024-11-18T18:36:55.381ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 1753455535, ip_addr: 104.131.155.175, stratum: 3, ref_time: 1731955013.3351338, correction: -0.412873179 } file = sled-agent/src/rack_setup/service.rs:740
8732024-11-18T18:36:55.381ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
8742024-11-18T18:36:55.397ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 12802533969288132606 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
8752024-11-18T18:36:55.397ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 9234514238289780840 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
8762024-11-18T18:36:56.472ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
8772024-11-18T18:36:56.472ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:45918
8782024-11-18T18:36:56.481ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 9031 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:45918 req_id = f13b9173-9d68-4087-b90c-786f297ca258 response_code = 200 uri = /timesync
8792024-11-18T18:36:56.485ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 1753455535, ip_addr: 104.131.155.175, stratum: 3, ref_time: 1731955014.419319, correction: -0.409531832 } file = sled-agent/src/rack_setup/service.rs:740
8802024-11-18T18:36:56.485ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
8812024-11-18T18:36:58.217ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
8822024-11-18T18:36:58.218ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:51044
8832024-11-18T18:36:58.226ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8609 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:51044 req_id = 97e656a3-1224-4175-a8bd-18b5a8fd55aa response_code = 200 uri = /timesync
8842024-11-18T18:36:58.230ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 1753455535, ip_addr: 104.131.155.175, stratum: 3, ref_time: 1731955016.8580222, correction: -0.404778332 } file = sled-agent/src/rack_setup/service.rs:740
8852024-11-18T18:36:58.230ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
8862024-11-18T18:37:01.954ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
8872024-11-18T18:37:01.954ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:58685
8882024-11-18T18:37:01.963ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8452 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:58685 req_id = 9ee9c688-57e6-4f28-849c-ff9ac6ff7126 response_code = 200 uri = /timesync
8892024-11-18T18:37:01.967ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 1753455535, ip_addr: 104.131.155.175, stratum: 3, ref_time: 1731955020.540668, correction: -0.394625723 } file = sled-agent/src/rack_setup/service.rs:740
8902024-11-18T18:37:01.967ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
8912024-11-18T18:37:02.477ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 128.13583ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
8922024-11-18T18:37:02.573ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 14973738965599029016 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
8932024-11-18T18:37:02.573ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 8425061366609949235 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
8942024-11-18T18:37:03.955ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
8952024-11-18T18:37:03.956ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:44563
8962024-11-18T18:37:03.962ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 17491455679030955142 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
8972024-11-18T18:37:03.962ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 9783823515617547413 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
8982024-11-18T18:37:03.965ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8944 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:44563 req_id = ca91ecb9-f67c-42d8-ba6b-39c7f59903d6 response_code = 200 uri = /timesync
8992024-11-18T18:37:03.968ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 1753455535, ip_addr: 104.131.155.175, stratum: 3, ref_time: 1731955022.9625857, correction: -0.389222264 } file = sled-agent/src/rack_setup/service.rs:740
9002024-11-18T18:37:03.968ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
9012024-11-18T18:37:04.552ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 4297017578838306490 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9022024-11-18T18:37:04.727ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 8291995201846061523 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9032024-11-18T18:37:05.234ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 13979037378220380202 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9042024-11-18T18:37:05.371ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 12802533969288132606 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9052024-11-18T18:37:05.372ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 9234514238289780840 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9062024-11-18T18:37:07.552ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
9072024-11-18T18:37:07.552ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:61841
9082024-11-18T18:37:07.561ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 9332 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:61841 req_id = dc0e6c6d-13ee-43bb-ad64-8eee2798e97c response_code = 200 uri = /timesync
9092024-11-18T18:37:07.564ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 1753455535, ip_addr: 104.131.155.175, stratum: 3, ref_time: 1731955026.223765, correction: -0.379437596 } file = sled-agent/src/rack_setup/service.rs:740
9102024-11-18T18:37:07.564ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
9112024-11-18T18:37:11.539ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
9122024-11-18T18:37:11.539ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:59668
9132024-11-18T18:37:11.549ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 9392 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:59668 req_id = dc6b01e4-a17f-4486-867a-2dc1f48be869 response_code = 200 uri = /timesync
9142024-11-18T18:37:11.551ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 1753455535, ip_addr: 104.131.155.175, stratum: 3, ref_time: 1731955030.284804, correction: -0.368617773 } file = sled-agent/src/rack_setup/service.rs:740
9152024-11-18T18:37:11.551ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
9162024-11-18T18:37:12.548ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 14973738965599029016 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9172024-11-18T18:37:12.549ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 8425061366609949235 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9182024-11-18T18:37:13.936ZERROSledAgent (kstat-sampler-worker): expiring kstat after too many failures error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:485 id = 17491455679030955142 reason = Duration(60s) sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9192024-11-18T18:37:13.937ZERROSledAgent (kstat-sampler-worker): expiring kstat after too many failures error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:485 id = 9783823515617547413 reason = Duration(60s) sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9202024-11-18T18:37:14.526ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 4297017578838306490 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9212024-11-18T18:37:14.701ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 8291995201846061523 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9222024-11-18T18:37:15.210ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 13979037378220380202 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9232024-11-18T18:37:15.345ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 12802533969288132606 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9242024-11-18T18:37:15.346ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 9234514238289780840 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9252024-11-18T18:37:16.721ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
9262024-11-18T18:37:16.721ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:36839
9272024-11-18T18:37:16.730ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 9391 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:36839 req_id = fb0b1e30-be52-4d87-9326-f84d77d4440e response_code = 200 uri = /timesync
9282024-11-18T18:37:16.733ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 1753455535, ip_addr: 104.131.155.175, stratum: 3, ref_time: 1731955035.2650988, correction: -0.354540795 } file = sled-agent/src/rack_setup/service.rs:740
9292024-11-18T18:37:16.733ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
9302024-11-18T18:37:17.599ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 722.625326ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9312024-11-18T18:37:22.523ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 14973738965599029016 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9322024-11-18T18:37:22.524ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 8425061366609949235 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9332024-11-18T18:37:23.404ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
9342024-11-18T18:37:23.404ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:60780
9352024-11-18T18:37:23.413ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8725 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:60780 req_id = bb7e5135-e058-4bc3-971b-39ddf0cff23d response_code = 200 uri = /timesync
9362024-11-18T18:37:23.416ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 1753455535, ip_addr: 104.131.155.175, stratum: 3, ref_time: 1731955041.200672, correction: -0.336335957 } file = sled-agent/src/rack_setup/service.rs:740
9372024-11-18T18:37:23.416ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
9382024-11-18T18:37:24.374ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:63039
9392024-11-18T18:37:24.374ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 34 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:63039 req_id = e792e31c-3236-496b-b717-ad3990be52a1 response_code = 200 uri = /baseboard
9402024-11-18T18:37:24.501ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 4297017578838306490 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9412024-11-18T18:37:24.675ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 8291995201846061523 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9422024-11-18T18:37:25.185ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 13979037378220380202 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9432024-11-18T18:37:25.319ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 12802533969288132606 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9442024-11-18T18:37:25.320ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 9234514238289780840 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9452024-11-18T18:37:29.917ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
9462024-11-18T18:37:29.917ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:47909
9472024-11-18T18:37:29.926ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 9302 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:47909 req_id = 20725d2a-6b2b-4742-a343-e52ddc38179c response_code = 200 uri = /timesync
9482024-11-18T18:37:29.930ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 1753455535, ip_addr: 104.131.155.175, stratum: 3, ref_time: 1731955047.2535858, correction: -0.318633139 } file = sled-agent/src/rack_setup/service.rs:740
9492024-11-18T18:37:29.930ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
9502024-11-18T18:37:32.499ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 14973738965599029016 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9512024-11-18T18:37:32.499ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 8425061366609949235 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9522024-11-18T18:37:33.316ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 973.440999ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9532024-11-18T18:37:34.476ZERROSledAgent (kstat-sampler-worker): expiring kstat after too many failures error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:485 id = 4297017578838306490 reason = Duration(60s) sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9542024-11-18T18:37:34.649ZERROSledAgent (kstat-sampler-worker): expiring kstat after too many failures error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:485 id = 8291995201846061523 reason = Duration(60s) sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9552024-11-18T18:37:35.160ZERROSledAgent (kstat-sampler-worker): expiring kstat after too many failures error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:485 id = 13979037378220380202 reason = Duration(60s) sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9562024-11-18T18:37:35.293ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 12802533969288132606 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9572024-11-18T18:37:35.294ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 9234514238289780840 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9582024-11-18T18:37:41.624ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
9592024-11-18T18:37:41.625ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:35847
9602024-11-18T18:37:41.634ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 9080 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:35847 req_id = 837fcd74-ec65-4f3f-a93b-a4bdd5486794 response_code = 200 uri = /timesync
9612024-11-18T18:37:41.639ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 1753455535, ip_addr: 104.131.155.175, stratum: 3, ref_time: 1731955059.3481574, correction: -0.286815256 } file = sled-agent/src/rack_setup/service.rs:740
9622024-11-18T18:37:41.639ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
9632024-11-18T18:37:42.474ZERROSledAgent (kstat-sampler-worker): expiring kstat after too many failures error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:485 id = 14973738965599029016 reason = Duration(60s) sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9642024-11-18T18:37:42.475ZERROSledAgent (kstat-sampler-worker): expiring kstat after too many failures error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:485 id = 8425061366609949235 reason = Duration(60s) sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9652024-11-18T18:37:45.268ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 12802533969288132606 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9662024-11-18T18:37:45.269ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 9234514238289780840 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9672024-11-18T18:37:49.281ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 2.20593772s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9682024-11-18T18:37:52.204ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
9692024-11-18T18:37:52.205ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:47375
9702024-11-18T18:37:52.214ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 9070 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:47375 req_id = 6221cfd3-5243-46c0-912a-8af4a9b59c8d response_code = 200 uri = /timesync
9712024-11-18T18:37:52.217ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 1753455535, ip_addr: 104.131.155.175, stratum: 3, ref_time: 1731955067.1442883, correction: -0.258078992 } file = sled-agent/src/rack_setup/service.rs:740
9722024-11-18T18:37:52.217ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
9732024-11-18T18:37:54.330ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:35131
9742024-11-18T18:37:54.330ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 34 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:35131 req_id = 57e4eefb-c862-43a4-9e54-2aff10348fb0 response_code = 200 uri = /baseboard
9752024-11-18T18:37:55.242ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 12802533969288132606 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9762024-11-18T18:37:55.242ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 9234514238289780840 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9772024-11-18T18:38:04.504ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
9782024-11-18T18:38:04.504ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:50235
9792024-11-18T18:38:04.513ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8743 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:50235 req_id = a674aeb6-b050-4ece-b6c0-93396168045f response_code = 200 uri = /timesync
9802024-11-18T18:38:04.516ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 1753455535, ip_addr: 104.131.155.175, stratum: 3, ref_time: 1731955083.993953, correction: -0.224706337 } file = sled-agent/src/rack_setup/service.rs:740
9812024-11-18T18:38:04.516ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
9822024-11-18T18:38:05.218ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 12802533969288132606 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9832024-11-18T18:38:05.218ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 9234514238289780840 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9842024-11-18T18:38:06.476ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 4.353808193s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9852024-11-18T18:38:15.192ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 12802533969288132606 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9862024-11-18T18:38:15.192ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 9234514238289780840 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9872024-11-18T18:38:19.949ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
9882024-11-18T18:38:19.949ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:38997
9892024-11-18T18:38:19.958ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 9249 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:38997 req_id = 9b2aa1c3-0118-474b-a121-f348b84d2bb7 response_code = 200 uri = /timesync
9902024-11-18T18:38:19.961ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 1753455535, ip_addr: 104.131.155.175, stratum: 3, ref_time: 1731955091.5612974, correction: -0.182761326 } file = sled-agent/src/rack_setup/service.rs:740
9912024-11-18T18:38:19.961ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
9922024-11-18T18:38:24.287ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:36722
9932024-11-18T18:38:24.287ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 48 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:36722 req_id = 4363a140-c644-43ee-ae50-f2992649160a response_code = 200 uri = /baseboard
9942024-11-18T18:38:25.167ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 12802533969288132606 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9952024-11-18T18:38:25.167ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 9234514238289780840 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9962024-11-18T18:38:25.813ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 8.702296757s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9972024-11-18T18:38:35.142ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 12802533969288132606 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9982024-11-18T18:38:35.142ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 9234514238289780840 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
9992024-11-18T18:38:37.203ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
10002024-11-18T18:38:37.204ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:45740
10012024-11-18T18:38:37.213ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 9472 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:45740 req_id = 227ce0b8-6a4d-4a15-840f-fca227c856ff response_code = 200 uri = /timesync
10022024-11-18T18:38:37.216ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 1753455535, ip_addr: 104.131.155.175, stratum: 3, ref_time: 1731955116.052057, correction: -0.135962382 } file = sled-agent/src/rack_setup/service.rs:740
10032024-11-18T18:38:37.216ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
10042024-11-18T18:38:45.118ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 12802533969288132606 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
10052024-11-18T18:38:45.118ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 9234514238289780840 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
10062024-11-18T18:38:49.487ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 10.137509955s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
10072024-11-18T18:38:50.599ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
10082024-11-18T18:38:50.599ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:50940
10092024-11-18T18:38:50.609ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 9534 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:50940 req_id = 7b456eb9-9f31-44fa-b810-3720e1ead315 response_code = 200 uri = /timesync
10102024-11-18T18:38:50.612ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 1753455535, ip_addr: 104.131.155.175, stratum: 3, ref_time: 1731955123.8516798, correction: -0.09958519 } file = sled-agent/src/rack_setup/service.rs:740
10112024-11-18T18:38:50.612ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
10122024-11-18T18:38:54.238ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:42942
10132024-11-18T18:38:54.238ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 43 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:42942 req_id = 8a8a76e8-6adc-415c-a51a-725668b70505 response_code = 200 uri = /baseboard
10142024-11-18T18:38:55.094ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 12802533969288132606 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
10152024-11-18T18:38:55.094ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 9234514238289780840 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
10162024-11-18T18:39:05.068ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 12802533969288132606 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
10172024-11-18T18:39:05.068ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 9234514238289780840 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
10182024-11-18T18:39:14.590ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 45.772781518s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
10192024-11-18T18:39:15.042ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 12802533969288132606 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
10202024-11-18T18:39:15.043ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 9234514238289780840 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
10212024-11-18T18:39:19.357ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
10222024-11-18T18:39:19.358ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:52608
10232024-11-18T18:39:19.379ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 21727 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:52608 req_id = 54c5ce37-7ebb-482c-b9ef-c55d58f6d53e response_code = 200 uri = /timesync
10242024-11-18T18:39:19.382ZWARNSledAgent (kstat-sampler-worker): replacing expired kstat target error = NoSuchKstat expiration_reason = Duration(60s) expired_at = 2024-11-18T18:37:34.649516669Z file = oximeter/instruments/src/kstat/sampler.rs:1028 id = 8291995201846061523 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
10252024-11-18T18:39:19.382ZWARNSledAgent (kstat-sampler-worker): replacing expired kstat target error = NoSuchKstat expiration_reason = Duration(60s) expired_at = 2024-11-18T18:37:35.160455391Z file = oximeter/instruments/src/kstat/sampler.rs:1028 id = 13979037378220380202 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
10262024-11-18T18:39:19.382ZWARNSledAgent (kstat-sampler-worker): replacing expired kstat target error = NoSuchKstat expiration_reason = Duration(60s) expired_at = 2024-11-18T18:37:42.474728145Z file = oximeter/instruments/src/kstat/sampler.rs:1028 id = 14973738965599029016 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
10272024-11-18T18:39:19.382ZWARNSledAgent (kstat-sampler-worker): replacing expired kstat target error = NoSuchKstat expiration_reason = Duration(60s) expired_at = 2024-11-18T18:37:34.476075674Z file = oximeter/instruments/src/kstat/sampler.rs:1028 id = 4297017578838306490 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
10282024-11-18T18:39:19.383ZWARNSledAgent (kstat-sampler-worker): replacing expired kstat target error = NoSuchKstat expiration_reason = Duration(60s) expired_at = 2024-11-18T18:37:42.475078952Z file = oximeter/instruments/src/kstat/sampler.rs:1028 id = 8425061366609949235 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
10292024-11-18T18:39:19.383ZWARNSledAgent (kstat-sampler-worker): replacing expired kstat target error = NoSuchKstat expiration_reason = Duration(60s) expired_at = 2024-11-18T18:37:13.936848451Z file = oximeter/instruments/src/kstat/sampler.rs:1028 id = 17491455679030955142 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
10302024-11-18T18:39:19.383ZWARNSledAgent (kstat-sampler-worker): replacing expired kstat target error = NoSuchKstat expiration_reason = Duration(60s) expired_at = 2024-11-18T18:37:13.937488264Z file = oximeter/instruments/src/kstat/sampler.rs:1028 id = 9783823515617547413 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
10312024-11-18T18:39:19.384ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: true, ref_id: 1753455535, ip_addr: 104.131.155.175, stratum: 3, ref_time: 1731955157.094909, correction: -0.021478664 } file = sled-agent/src/rack_setup/service.rs:740
10322024-11-18T18:39:19.384ZINFOSledAgent (RSS): Finished setting up Internal DNS and NTP file = sled-agent/src/rack_setup/service.rs:1425
10332024-11-18T18:39:19.414ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 55213fec-a100-46b7-9721-e06d9201c8d5 (service), filesystem_pool: Some(ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 07f4fc36-9902-4aeb-a7d9-f99de01ff5c2 (service), filesystem_pool: Some(ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: b160705f-2e55-4b76-ba79-187b4901f005 (service), filesystem_pool: Some(ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 6dfcb5a1-e0b1-44a6-97dc-3d4ea97e4f50 (service), filesystem_pool: Some(ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: c9bbb6b8-8989-4090-b169-2e9a21574d6c, kind: Service { id: 6dfcb5a1-e0b1-44a6-97dc-3d4ea97e4f50 }, name: Name("ntp-6dfcb5a1-e0b1-44a6-97dc-3d4ea97e4f50"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 148, 216])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 9d6a1714-4692-4a2d-9475-dc5b067334ad (service), filesystem_pool: Some(ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External } } } }, OmicronZoneConfig { id: e0a33abb-5d16-47b8-a088-6370c29fad2a (service), filesystem_pool: Some(ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External } } } }, OmicronZoneConfig { id: 509f6459-b46e-474b-8247-9cd9c287fa80 (service), filesystem_pool: Some(ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External } } } }, OmicronZoneConfig { id: d164bfd1-7ae2-47c5-bc53-968633c77a9d (service), filesystem_pool: Some(ZpoolName { id: 0199d4fd-4090-4a7c-83e6-040814eb79bb (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0199d4fd-4090-4a7c-83e6-040814eb79bb (zpool), kind: External } } } }, OmicronZoneConfig { id: b9daa5c1-3421-4e6b-889a-4c4b145361da (service), filesystem_pool: Some(ZpoolName { id: 5840d858-3782-45e1-a6e0-d60d070ad2c3 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 5840d858-3782-45e1-a6e0-d60d070ad2c3 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:538 sled_address = [fd00:1122:3344:101::1]:12345
10342024-11-18T18:39:19.414ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:55744
10352024-11-18T18:39:19.415ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 55213fec-a100-46b7-9721-e06d9201c8d5, dataset_name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
10362024-11-18T18:39:19.415ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 55213fec-a100-46b7-9721-e06d9201c8d5, dataset_name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: InternalDns }, responder: ... }
10372024-11-18T18:39:19.455ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 07f4fc36-9902-4aeb-a7d9-f99de01ff5c2, dataset_name: DatasetName { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
10382024-11-18T18:39:19.455ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 07f4fc36-9902-4aeb-a7d9-f99de01ff5c2, dataset_name: DatasetName { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, kind: InternalDns }, responder: ... }
10392024-11-18T18:39:19.486ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: b160705f-2e55-4b76-ba79-187b4901f005, dataset_name: DatasetName { pool_name: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
10402024-11-18T18:39:19.486ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: b160705f-2e55-4b76-ba79-187b4901f005, dataset_name: DatasetName { pool_name: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }, kind: InternalDns }, responder: ... }
10412024-11-18T18:39:19.519ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 9d6a1714-4692-4a2d-9475-dc5b067334ad, dataset_name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
10422024-11-18T18:39:19.519ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 9d6a1714-4692-4a2d-9475-dc5b067334ad, dataset_name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10432024-11-18T18:39:19.551ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: e0a33abb-5d16-47b8-a088-6370c29fad2a, dataset_name: DatasetName { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
10442024-11-18T18:39:19.551ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: e0a33abb-5d16-47b8-a088-6370c29fad2a, dataset_name: DatasetName { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10452024-11-18T18:39:19.583ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 509f6459-b46e-474b-8247-9cd9c287fa80, dataset_name: DatasetName { pool_name: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
10462024-11-18T18:39:19.583ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 509f6459-b46e-474b-8247-9cd9c287fa80, dataset_name: DatasetName { pool_name: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10472024-11-18T18:39:19.615ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: d164bfd1-7ae2-47c5-bc53-968633c77a9d, dataset_name: DatasetName { pool_name: ZpoolName { id: 0199d4fd-4090-4a7c-83e6-040814eb79bb (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
10482024-11-18T18:39:19.615ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: d164bfd1-7ae2-47c5-bc53-968633c77a9d, dataset_name: DatasetName { pool_name: ZpoolName { id: 0199d4fd-4090-4a7c-83e6-040814eb79bb (zpool), kind: External }, kind: Cockroach }, responder: ... }
10492024-11-18T18:39:19.646ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: b9daa5c1-3421-4e6b-889a-4c4b145361da, dataset_name: DatasetName { pool_name: ZpoolName { id: 5840d858-3782-45e1-a6e0-d60d070ad2c3 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
10502024-11-18T18:39:19.646ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: b9daa5c1-3421-4e6b-889a-4c4b145361da, dataset_name: DatasetName { pool_name: ZpoolName { id: 5840d858-3782-45e1-a6e0-d60d070ad2c3 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10512024-11-18T18:39:19.677ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10522024-11-18T18:39:19.677ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/9216f0f9-29c9-4d33-8c71-0f237e1331a0/config/omicron-zones.json file = common/src/ledger.rs:177
10532024-11-18T18:39:19.677ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/c23ca4a6-dcd8-4835-a278-26664c492a4d/config/omicron-zones.json file = common/src/ledger.rs:177
10542024-11-18T18:39:19.678ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10552024-11-18T18:39:19.762ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10562024-11-18T18:39:19.762ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10572024-11-18T18:39:19.762ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10582024-11-18T18:39:19.762ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10592024-11-18T18:39:19.762ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10602024-11-18T18:39:19.855ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_d164bfd1-7ae2-47c5-bc53-968633c77a9d file = illumos-utils/src/zone.rs:325
10612024-11-18T18:39:19.859ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_509f6459-b46e-474b-8247-9cd9c287fa80 file = illumos-utils/src/zone.rs:325
10622024-11-18T18:39:19.864ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_b9daa5c1-3421-4e6b-889a-4c4b145361da file = illumos-utils/src/zone.rs:325
10632024-11-18T18:39:19.869ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a file = illumos-utils/src/zone.rs:325
10642024-11-18T18:39:19.873ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_9d6a1714-4692-4a2d-9475-dc5b067334ad file = illumos-utils/src/zone.rs:325
10652024-11-18T18:39:19.879ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_d164bfd1-7ae2-47c5-bc53-968633c77a9d file = illumos-utils/src/zone.rs:363
10662024-11-18T18:39:19.884ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_509f6459-b46e-474b-8247-9cd9c287fa80 file = illumos-utils/src/zone.rs:363
10672024-11-18T18:39:19.888ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_b9daa5c1-3421-4e6b-889a-4c4b145361da file = illumos-utils/src/zone.rs:363
10682024-11-18T18:39:19.893ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a file = illumos-utils/src/zone.rs:363
10692024-11-18T18:39:19.898ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_9d6a1714-4692-4a2d-9475-dc5b067334ad file = illumos-utils/src/zone.rs:363
10702024-11-18T18:39:22.718ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_d164bfd1-7ae2-47c5-bc53-968633c77a9d: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService5'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::6]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='d164bfd1-7ae2-47c5-bc53-968633c77a9d'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::6]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::6]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10712024-11-18T18:39:22.718ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_d164bfd1-7ae2-47c5-bc53-968633c77a9d zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_d164bfd1-7ae2-47c5-bc53-968633c77a9d
10722024-11-18T18:39:22.743ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_b9daa5c1-3421-4e6b-889a-4c4b145361da: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService7'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::7'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::7]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='b9daa5c1-3421-4e6b-889a-4c4b145361da'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::7]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::7]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10732024-11-18T18:39:22.744ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_b9daa5c1-3421-4e6b-889a-4c4b145361da zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_b9daa5c1-3421-4e6b-889a-4c4b145361da
10742024-11-18T18:39:22.839ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_509f6459-b46e-474b-8247-9cd9c287fa80: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService6'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::5]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='509f6459-b46e-474b-8247-9cd9c287fa80'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::5]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::5]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10752024-11-18T18:39:22.840ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_509f6459-b46e-474b-8247-9cd9c287fa80 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_509f6459-b46e-474b-8247-9cd9c287fa80
10762024-11-18T18:39:22.861ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d164bfd1-7ae2-47c5-bc53-968633c77a9d") failed: Property not found. retry in 29.452994ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d164bfd1-7ae2-47c5-bc53-968633c77a9d
10772024-11-18T18:39:22.872ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService8'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::4'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::4]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='e0a33abb-5d16-47b8-a088-6370c29fad2a'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::4]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::4]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10782024-11-18T18:39:22.872ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a
10792024-11-18T18:39:22.897ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b9daa5c1-3421-4e6b-889a-4c4b145361da") failed: Property not found. retry in 53.707695ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b9daa5c1-3421-4e6b-889a-4c4b145361da
10802024-11-18T18:39:22.905ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d164bfd1-7ae2-47c5-bc53-968633c77a9d") failed: Property not found. retry in 141.1854ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d164bfd1-7ae2-47c5-bc53-968633c77a9d
10812024-11-18T18:39:22.906ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_9d6a1714-4692-4a2d-9475-dc5b067334ad: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService9'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::3'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::3]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='9d6a1714-4692-4a2d-9475-dc5b067334ad'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::3]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::3]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10822024-11-18T18:39:22.906ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_9d6a1714-4692-4a2d-9475-dc5b067334ad zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_9d6a1714-4692-4a2d-9475-dc5b067334ad
10832024-11-18T18:39:22.968ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b9daa5c1-3421-4e6b-889a-4c4b145361da") failed: Property not found. retry in 124.787017ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b9daa5c1-3421-4e6b-889a-4c4b145361da
10842024-11-18T18:39:23.163ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d164bfd1-7ae2-47c5-bc53-968633c77a9d") failed: Property not found. retry in 131.712355ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d164bfd1-7ae2-47c5-bc53-968633c77a9d
10852024-11-18T18:39:23.487ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b9daa5c1-3421-4e6b-889a-4c4b145361da") failed: Property not found. retry in 175.397663ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b9daa5c1-3421-4e6b-889a-4c4b145361da
10862024-11-18T18:39:23.509ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d164bfd1-7ae2-47c5-bc53-968633c77a9d") failed: Property not found. retry in 232.346945ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d164bfd1-7ae2-47c5-bc53-968633c77a9d
10872024-11-18T18:39:23.559ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9d6a1714-4692-4a2d-9475-dc5b067334ad") failed: Property not found. retry in 38.744663ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9d6a1714-4692-4a2d-9475-dc5b067334ad
10882024-11-18T18:39:23.573ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_509f6459-b46e-474b-8247-9cd9c287fa80") failed: Property not found. retry in 42.95797ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_509f6459-b46e-474b-8247-9cd9c287fa80
10892024-11-18T18:39:23.591ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a") failed: Property not found. retry in 32.00267ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a
10902024-11-18T18:39:23.621ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9d6a1714-4692-4a2d-9475-dc5b067334ad") failed: Property not found. retry in 102.071698ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9d6a1714-4692-4a2d-9475-dc5b067334ad
10912024-11-18T18:39:23.655ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_509f6459-b46e-474b-8247-9cd9c287fa80") failed: Property not found. retry in 77.80944ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_509f6459-b46e-474b-8247-9cd9c287fa80
10922024-11-18T18:39:23.699ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a") failed: Property not found. retry in 146.105113ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a
10932024-11-18T18:39:23.993ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b9daa5c1-3421-4e6b-889a-4c4b145361da") failed: Property not found. retry in 548.412636ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b9daa5c1-3421-4e6b-889a-4c4b145361da
10942024-11-18T18:39:24.094ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a") failed: Property not found. retry in 249.818866ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a
10952024-11-18T18:39:24.357ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9d6a1714-4692-4a2d-9475-dc5b067334ad") failed: Property not found. retry in 210.46939ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9d6a1714-4692-4a2d-9475-dc5b067334ad
10962024-11-18T18:39:24.694ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d164bfd1-7ae2-47c5-bc53-968633c77a9d") failed: Property not found. retry in 642.338538ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d164bfd1-7ae2-47c5-bc53-968633c77a9d
10972024-11-18T18:39:24.731ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_509f6459-b46e-474b-8247-9cd9c287fa80") failed: Property not found. retry in 169.896702ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_509f6459-b46e-474b-8247-9cd9c287fa80
10982024-11-18T18:39:24.803ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b9daa5c1-3421-4e6b-889a-4c4b145361da") failed: Property not found. retry in 965.938768ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b9daa5c1-3421-4e6b-889a-4c4b145361da
10992024-11-18T18:39:24.947ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a") failed: Property not found. retry in 378.533564ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a
11002024-11-18T18:39:25.057ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9d6a1714-4692-4a2d-9475-dc5b067334ad") failed: Property not found. retry in 360.658516ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9d6a1714-4692-4a2d-9475-dc5b067334ad
11012024-11-18T18:39:25.216ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_509f6459-b46e-474b-8247-9cd9c287fa80") failed: Property not found. retry in 251.012598ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_509f6459-b46e-474b-8247-9cd9c287fa80
11022024-11-18T18:39:25.308ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:37808
11032024-11-18T18:39:25.308ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 116 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:37808 req_id = 41415439-c6e0-458f-b17f-e2a319c5b422 response_code = 200 uri = /baseboard
11042024-11-18T18:39:25.384ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a") failed: Property not found. retry in 587.321927ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a
11052024-11-18T18:39:25.402ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d164bfd1-7ae2-47c5-bc53-968633c77a9d") failed: Property not found. retry in 1.233175193s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d164bfd1-7ae2-47c5-bc53-968633c77a9d
11062024-11-18T18:39:25.499ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9d6a1714-4692-4a2d-9475-dc5b067334ad") failed: Property not found. retry in 1.096387508s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9d6a1714-4692-4a2d-9475-dc5b067334ad
11072024-11-18T18:39:25.570ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_509f6459-b46e-474b-8247-9cd9c287fa80") failed: Property not found. retry in 673.153481ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_509f6459-b46e-474b-8247-9cd9c287fa80
11082024-11-18T18:39:25.781ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b9daa5c1-3421-4e6b-889a-4c4b145361da") failed: Property not found. retry in 897.752039ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b9daa5c1-3421-4e6b-889a-4c4b145361da
11092024-11-18T18:39:25.982ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a") failed: Property not found. retry in 852.719631ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a
11102024-11-18T18:39:26.253ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_509f6459-b46e-474b-8247-9cd9c287fa80") failed: Property not found. retry in 1.123628733s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_509f6459-b46e-474b-8247-9cd9c287fa80
11112024-11-18T18:39:26.603ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9d6a1714-4692-4a2d-9475-dc5b067334ad") failed: Property not found. retry in 949.091693ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9d6a1714-4692-4a2d-9475-dc5b067334ad
11122024-11-18T18:39:26.642ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d164bfd1-7ae2-47c5-bc53-968633c77a9d") failed: Property not found. retry in 608.930189ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d164bfd1-7ae2-47c5-bc53-968633c77a9d
11132024-11-18T18:39:26.685ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b9daa5c1-3421-4e6b-889a-4c4b145361da") failed: Property not found. retry in 543.50336ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b9daa5c1-3421-4e6b-889a-4c4b145361da
11142024-11-18T18:39:26.842ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a") failed: Property not found. retry in 768.852196ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a
11152024-11-18T18:39:27.237ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b9daa5c1-3421-4e6b-889a-4c4b145361da") failed: Property not found. retry in 1.396193428s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b9daa5c1-3421-4e6b-889a-4c4b145361da
11162024-11-18T18:39:27.258ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d164bfd1-7ae2-47c5-bc53-968633c77a9d") failed: Property not found. retry in 924.218403ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d164bfd1-7ae2-47c5-bc53-968633c77a9d
11172024-11-18T18:39:27.383ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_509f6459-b46e-474b-8247-9cd9c287fa80") failed: Property not found. retry in 923.812485ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_509f6459-b46e-474b-8247-9cd9c287fa80
11182024-11-18T18:39:27.559ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9d6a1714-4692-4a2d-9475-dc5b067334ad") failed: Property not found. retry in 1.385366355s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9d6a1714-4692-4a2d-9475-dc5b067334ad
11192024-11-18T18:39:27.619ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a") failed: Property not found. retry in 846.50441ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a
11202024-11-18T18:39:28.197ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d164bfd1-7ae2-47c5-bc53-968633c77a9d") failed: Property not found. retry in 1.344978789s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d164bfd1-7ae2-47c5-bc53-968633c77a9d
11212024-11-18T18:39:28.318ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_509f6459-b46e-474b-8247-9cd9c287fa80") failed: Property not found. retry in 964.049236ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_509f6459-b46e-474b-8247-9cd9c287fa80
11222024-11-18T18:39:28.475ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a") failed: Property not found. retry in 855.026847ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a
11232024-11-18T18:39:28.644ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b9daa5c1-3421-4e6b-889a-4c4b145361da") failed: Property not found. retry in 624.043442ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b9daa5c1-3421-4e6b-889a-4c4b145361da
11242024-11-18T18:39:28.953ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9d6a1714-4692-4a2d-9475-dc5b067334ad") failed: Property not found. retry in 977.984703ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9d6a1714-4692-4a2d-9475-dc5b067334ad
11252024-11-18T18:39:29.277ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b9daa5c1-3421-4e6b-889a-4c4b145361da") failed: Property not found. retry in 678.922384ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b9daa5c1-3421-4e6b-889a-4c4b145361da
11262024-11-18T18:39:29.290ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_509f6459-b46e-474b-8247-9cd9c287fa80") failed: Property not found. retry in 962.012026ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_509f6459-b46e-474b-8247-9cd9c287fa80
11272024-11-18T18:39:29.338ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a") failed: Property not found. retry in 993.844923ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a
11282024-11-18T18:39:29.550ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d164bfd1-7ae2-47c5-bc53-968633c77a9d") failed: Property not found. retry in 987.233335ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d164bfd1-7ae2-47c5-bc53-968633c77a9d
11292024-11-18T18:39:29.940ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9d6a1714-4692-4a2d-9475-dc5b067334ad") failed: Property not found. retry in 1.170556427s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9d6a1714-4692-4a2d-9475-dc5b067334ad
11302024-11-18T18:39:29.965ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b9daa5c1-3421-4e6b-889a-4c4b145361da") failed: Property not found. retry in 1.0490205s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b9daa5c1-3421-4e6b-889a-4c4b145361da
11312024-11-18T18:39:30.262ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_509f6459-b46e-474b-8247-9cd9c287fa80") failed: Property not found. retry in 661.233426ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_509f6459-b46e-474b-8247-9cd9c287fa80
11322024-11-18T18:39:30.343ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a") failed: Property not found. retry in 1.300815489s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a
11332024-11-18T18:39:30.547ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d164bfd1-7ae2-47c5-bc53-968633c77a9d") failed: Property not found. retry in 1.334940549s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d164bfd1-7ae2-47c5-bc53-968633c77a9d
11342024-11-18T18:39:30.932ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_509f6459-b46e-474b-8247-9cd9c287fa80") failed: Property not found. retry in 512.589296ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_509f6459-b46e-474b-8247-9cd9c287fa80
11352024-11-18T18:39:31.023ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_b9daa5c1-3421-4e6b-889a-4c4b145361da") failed: Property not found. retry in 752.966214ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_b9daa5c1-3421-4e6b-889a-4c4b145361da
11362024-11-18T18:39:31.120ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9d6a1714-4692-4a2d-9475-dc5b067334ad") failed: Property not found. retry in 1.238024804s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9d6a1714-4692-4a2d-9475-dc5b067334ad
11372024-11-18T18:39:31.656ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a") failed: Property not found. retry in 1.435158049s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a
11382024-11-18T18:39:33.104ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a") failed: Property not found. retry in 1.244953703s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a
11392024-11-18T18:39:34.360ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a") failed: Property not found. retry in 607.911743ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a
11402024-11-18T18:39:34.982ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a") failed: Property not found. retry in 691.560518ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a
11412024-11-18T18:39:36.586ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a") failed: Property not found. retry in 1.420036573s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a
11422024-11-18T18:39:38.017ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a") failed: Property not found. retry in 1.121253274s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a
11432024-11-18T18:39:39.149ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a") failed: Property not found. retry in 1.122204267s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a
11442024-11-18T18:39:40.281ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a") failed: Property not found. retry in 829.741596ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a
11452024-11-18T18:39:41.120ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a") failed: Property not found. retry in 709.863916ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a
11462024-11-18T18:39:41.853ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3311 zone = oxz_cockroachdb_d164bfd1-7ae2-47c5-bc53-968633c77a9d
11472024-11-18T18:39:41.853ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3311 zone = oxz_cockroachdb_509f6459-b46e-474b-8247-9cd9c287fa80
11482024-11-18T18:39:41.853ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3311 zone = oxz_cockroachdb_b9daa5c1-3421-4e6b-889a-4c4b145361da
11492024-11-18T18:39:41.853ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3311 zone = oxz_cockroachdb_e0a33abb-5d16-47b8-a088-6370c29fad2a
11502024-11-18T18:39:41.853ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3311 zone = oxz_cockroachdb_9d6a1714-4692-4a2d-9475-dc5b067334ad
11512024-11-18T18:39:41.853ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/9216f0f9-29c9-4d33-8c71-0f237e1331a0/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
11522024-11-18T18:39:41.854ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/c23ca4a6-dcd8-4835-a278-26664c492a4d/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
11532024-11-18T18:39:41.857ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 22464115 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:55744 req_id = 34a75359-7a82-41be-ab62-ca205ddf7695 response_code = 204 uri = /omicron-zones
11542024-11-18T18:39:41.894ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:53325
11552024-11-18T18:39:41.928ZINFOSledAgent (ServiceManager): Initializing CRDB Cluster - sending request to [fd00:1122:3344:101::5]:32221 file = sled-agent/src/services.rs:3744
11562024-11-18T18:39:43.694ZINFOSledAgent (ServiceManager): Formatting CRDB file = sled-agent/src/services.rs:3762
11572024-11-18T18:39:54.574ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize CockroachDb: Error running command in zone 'oxz_cockroachdb_509f6459-b46e-474b-8247-9cd9c287fa80': Command [/opt/oxide/cockroachdb/bin/cockroach sql --insecure --host [fd00:1122:3344:101::5]:32221 --file /opt/oxide/cockroachdb/sql/dbinit.sql] executed and failed with status: exit status: 1 stdout: BEGIN\nCREATE DATABASE\nCREATE ROLE\nALTER DEFAULT PRIVILEGES\nCONFIGURE ZONE 1\nCREATE TYPE\nCREATE TABLE\nCREATE TABLE\nCREATE TYPE\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE TYPE\nCREATE TYPE\nCREATE TYPE\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE TABLE\nCREATE TABLE\nCREATE TABLE\nCREATE INDEX\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE TYPE\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE TABLE\nCREATE VIEW\nCREATE TABLE\nCREATE INDEX\nCREATE TYPE\nCREATE TYPE\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE VIEW\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE TABLE\nCREATE VIEW\nCREATE VIEW\nCREATE INDEX\nCREATE INDEX\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE VIEW\nCREATE VIEW\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE TYPE\nCREATE TYPE\nCREATE TYPE\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE TYPE\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE VIEW\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE TABLE\nCREATE TABLE\nCREATE TABLE\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nINSERT 1\nCREATE TABLE\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE TABLE\nCREATE TYPE\nCREATE TABLE\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE TYPE\nCREATE TABLE\nCREATE TYPE\nCREATE TYPE\nCREATE TABLE\nCREATE TABLE\nCREATE TABLE\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE TABLE\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE TABLE\nCREATE TABLE\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE TABLE\nCREATE TABLE\nCREATE TABLE\nCREATE INDEX\nCREATE TYPE\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE TYPE\nCREATE TABLE\nCREATE TYPE\nCREATE TABLE\nCREATE TYPE\nCREATE TABLE\nCREATE TYPE\nCREATE TABLE\nCREATE TYPE\nCREATE TABLE\nCREATE TABLE\nCREATE TABLE\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE TABLE\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE TABLE\nCREATE TYPE\nCREATE TYPE\nCREATE TABLE\nCREATE TABLE\nCREATE TABLE\nCREATE TABLE\nCREATE TABLE\nCREATE TABLE\nCREATE TABLE\nCREATE TABLE\nCREATE TABLE\nCREATE TABLE\nCREATE TABLE\nCREATE TABLE\nCREATE TABLE\nCREATE TABLE\nCOMMIT\nBEGIN\nCREATE TABLE\nCREATE INDEX\nCREATE VIEW\nCREATE SEQUENCE\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE VIEW\nCREATE TABLE\nCREATE INDEX\nCREATE TYPE\nCREATE TYPE\nCREATE TABLE\nCREATE TABLE\nCREATE TYPE\nCREATE TABLE\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE VIEW\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE TABLE\nCREATE TABLE\nINSERT 1\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nINSERT 1\n stderr: ERROR: transaction committed but schema change aborted with error: (53200): failed to ingest index entries during backfill: not enough memory available to create a BulkAdder: root: memory budget exceeded: 33554432 bytes requested, 101933056 currently allocated, 134217728 bytes in budget\nSQLSTATE: XXA00\nHINT: Consider increasing --max-sql-memory startup parameter.\n--\nTry setting a higher --max-sql-memory.\n--\nSome of the non-DDL statements may have committed successfully, but some of the DDL statement(s) failed.\nManual inspection may be required to determine the actual state of the database.\n--\nSee: https://github.com/cockroachdb/cockroach/issues/42061\nERROR: transaction committed but schema change aborted with error: (53200): failed to ingest index entries during backfill: not enough memory available to create a BulkAdder: root: memory budget exceeded: 33554432 bytes requested, 101933056 currently allocated, 134217728 bytes in budget\nSQLSTATE: XXA00\nHINT: Consider increasing --max-sql-memory startup parameter.\n--\nTry setting a higher --max-sql-memory.\n--\nSome of the non-DDL statements may have committed successfully, but some of the DDL statement(s) failed.\nManual inspection may be required to determine the actual state of the database.\n--\nSee: https://github.com/cockroachdb/cockroach/issues/42061\nFailed running "sql"\n file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 12679796 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:53325 req_id = bb53c61b-32f3-45a9-abcc-acbe6574c347 response_code = 500 uri = /cockroachdb
11582024-11-18T18:39:54.575ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "bb53c61b-32f3-45a9-abcc-acbe6574c347", "content-length": "124", "date": "Mon, 18 Nov 2024 18:39:54 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "bb53c61b-32f3-45a9-abcc-acbe6574c347" } file = sled-agent/src/rack_setup/service.rs:1154 retry_after = 121.948684ms
11592024-11-18T18:39:54.729ZINFOSledAgent (ServiceManager): Initializing CRDB Cluster - sending request to [fd00:1122:3344:101::5]:32221 file = sled-agent/src/services.rs:3744
11602024-11-18T18:39:54.849ZINFOSledAgent (ServiceManager): Formatting CRDB file = sled-agent/src/services.rs:3762
11612024-11-18T18:40:01.828ZINFOSledAgent (ServiceManager): Formatting CRDB - Completed file = sled-agent/src/services.rs:3785
11622024-11-18T18:40:01.828ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7129826 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:53325 req_id = 0537a8f2-1164-401d-b70e-ef3937eabc98 response_code = 204 uri = /cockroachdb
11632024-11-18T18:40:01.829ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:57668
11642024-11-18T18:40:01.829ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 148 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:57668 req_id = 1ce713c7-075f-47a8-8bcb-d03b33565a17 response_code = 200 uri = /baseboard
11652024-11-18T18:40:01.892ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(5), zones: [OmicronZoneConfig { id: 55213fec-a100-46b7-9721-e06d9201c8d5 (service), filesystem_pool: Some(ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 07f4fc36-9902-4aeb-a7d9-f99de01ff5c2 (service), filesystem_pool: Some(ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: b160705f-2e55-4b76-ba79-187b4901f005 (service), filesystem_pool: Some(ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 6dfcb5a1-e0b1-44a6-97dc-3d4ea97e4f50 (service), filesystem_pool: Some(ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: c9bbb6b8-8989-4090-b169-2e9a21574d6c, kind: Service { id: 6dfcb5a1-e0b1-44a6-97dc-3d4ea97e4f50 }, name: Name("ntp-6dfcb5a1-e0b1-44a6-97dc-3d4ea97e4f50"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 148, 216])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 9d6a1714-4692-4a2d-9475-dc5b067334ad (service), filesystem_pool: Some(ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External } } } }, OmicronZoneConfig { id: e0a33abb-5d16-47b8-a088-6370c29fad2a (service), filesystem_pool: Some(ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External } } } }, OmicronZoneConfig { id: 509f6459-b46e-474b-8247-9cd9c287fa80 (service), filesystem_pool: Some(ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External } } } }, OmicronZoneConfig { id: d164bfd1-7ae2-47c5-bc53-968633c77a9d (service), filesystem_pool: Some(ZpoolName { id: 0199d4fd-4090-4a7c-83e6-040814eb79bb (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0199d4fd-4090-4a7c-83e6-040814eb79bb (zpool), kind: External } } } }, OmicronZoneConfig { id: b9daa5c1-3421-4e6b-889a-4c4b145361da (service), filesystem_pool: Some(ZpoolName { id: 5840d858-3782-45e1-a6e0-d60d070ad2c3 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 5840d858-3782-45e1-a6e0-d60d070ad2c3 (zpool), kind: External } } } }, OmicronZoneConfig { id: 6767b855-63e1-4a7d-ac38-c4e7eb56747f (service), filesystem_pool: Some(ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }), zone_type: ExternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External } }, http_address: [fd00:1122:3344:101::8]:5353, dns_address: 10.151.2.164:53, nic: NetworkInterface { id: 6a2febaf-2bbc-4cc0-aab6-80daf994ec33, kind: Service { id: 6767b855-63e1-4a7d-ac38-c4e7eb56747f }, name: Name("external-dns-6767b855-63e1-4a7d-ac38-c4e7eb56747f"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 152, 29])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } } }, OmicronZoneConfig { id: ceea2287-3571-41ea-8f15-b2aeb4a74fd9 (service), filesystem_pool: Some(ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }), zone_type: ExternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External } }, http_address: [fd00:1122:3344:101::9]:5353, dns_address: 10.151.2.165:53, nic: NetworkInterface { id: 72590ac9-2812-4929-9b37-1829b4168efa, kind: Service { id: ceea2287-3571-41ea-8f15-b2aeb4a74fd9 }, name: Name("external-dns-ceea2287-3571-41ea-8f15-b2aeb4a74fd9"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 170, 19])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } } }, OmicronZoneConfig { id: 1bd1422d-700f-41e4-b299-7268d8edd3b1 (service), filesystem_pool: Some(ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::a]:12221, external_ip: 10.151.2.166, nic: NetworkInterface { id: f6afceac-9bde-4010-ba9d-6c2eebc86b2d, kind: Service { id: 1bd1422d-700f-41e4-b299-7268d8edd3b1 }, name: Name("nexus-1bd1422d-700f-41e4-b299-7268d8edd3b1"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 0])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: 88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d (service), filesystem_pool: Some(ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::b]:12221, external_ip: 10.151.2.167, nic: NetworkInterface { id: d0f6d114-ab1d-41fb-a2bb-c154f579e21f, kind: Service { id: 88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d }, name: Name("nexus-88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 37])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: d7ddf21e-0e1e-4004-b302-ecb81ba84c09 (service), filesystem_pool: Some(ZpoolName { id: 453f7245-5f70-4fe8-b37e-054e56d5c1c5 (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::c]:12221, external_ip: 10.151.2.168, nic: NetworkInterface { id: ab6c5d8c-f7d0-4cf6-8c3c-5acf8f3959a9, kind: Service { id: d7ddf21e-0e1e-4004-b302-ecb81ba84c09 }, name: Name("nexus-d7ddf21e-0e1e-4004-b302-ecb81ba84c09"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 151, 123])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: 8716f96b-ee60-45b3-a886-a68fcf5530d6 (service), filesystem_pool: Some(ZpoolName { id: 453f7245-5f70-4fe8-b37e-054e56d5c1c5 (zpool), kind: External }), zone_type: Oximeter { address: [fd00:1122:3344:101::d]:12223 } }, OmicronZoneConfig { id: 55ab4bd1-3e97-453c-b90c-176360dea573 (service), filesystem_pool: Some(ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }), zone_type: Clickhouse { address: [fd00:1122:3344:101::e]:8123, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External } } } }, OmicronZoneConfig { id: 1fa25dd7-d8f9-49ef-bb20-c6bad413aace (service), filesystem_pool: Some(ZpoolName { id: 131b4d69-dbb0-4397-9284-176e4a3c973d (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::f]:17000 } }, OmicronZoneConfig { id: 023a09b8-2c2b-483c-be8c-540d1a35ce5a (service), filesystem_pool: Some(ZpoolName { id: 5840d858-3782-45e1-a6e0-d60d070ad2c3 (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::10]:17000 } }, OmicronZoneConfig { id: b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46 (service), filesystem_pool: Some(ZpoolName { id: 0199d4fd-4090-4a7c-83e6-040814eb79bb (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::11]:17000 } }, OmicronZoneConfig { id: 396921f6-50a5-459a-86ad-182888d80a34 (service), filesystem_pool: Some(ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::12]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External } } } }, OmicronZoneConfig { id: efcbb893-1918-43b9-8315-e9bb90d664e8 (service), filesystem_pool: Some(ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::13]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External } } } }, OmicronZoneConfig { id: 6062ae2a-331c-4cb6-a9b5-eb4310b33530 (service), filesystem_pool: Some(ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::14]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External } } } }, OmicronZoneConfig { id: bb775b2d-b688-494a-8509-79ecf4eb8f60 (service), filesystem_pool: Some(ZpoolName { id: 0199d4fd-4090-4a7c-83e6-040814eb79bb (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::15]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0199d4fd-4090-4a7c-83e6-040814eb79bb (zpool), kind: External } } } }, OmicronZoneConfig { id: e782f5fc-076e-414c-b87e-6b09d1bdfaeb (service), filesystem_pool: Some(ZpoolName { id: 5840d858-3782-45e1-a6e0-d60d070ad2c3 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::16]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 5840d858-3782-45e1-a6e0-d60d070ad2c3 (zpool), kind: External } } } }, OmicronZoneConfig { id: 3aad33dc-2f67-4a47-b973-90de986a5f65 (service), filesystem_pool: Some(ZpoolName { id: 131b4d69-dbb0-4397-9284-176e4a3c973d (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::17]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 131b4d69-dbb0-4397-9284-176e4a3c973d (zpool), kind: External } } } }, OmicronZoneConfig { id: 81488621-5898-41d4-a112-d3e574fb2d33 (service), filesystem_pool: Some(ZpoolName { id: 453f7245-5f70-4fe8-b37e-054e56d5c1c5 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::18]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 453f7245-5f70-4fe8-b37e-054e56d5c1c5 (zpool), kind: External } } } }, OmicronZoneConfig { id: 2735d4d6-073c-4860-ae50-e3742cc2cefa (service), filesystem_pool: Some(ZpoolName { id: b09b6e80-5341-4ded-a520-3f312403e198 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::19]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: b09b6e80-5341-4ded-a520-3f312403e198 (zpool), kind: External } } } }, OmicronZoneConfig { id: 10904f20-d090-41dc-9a54-dab632f01d4a (service), filesystem_pool: Some(ZpoolName { id: f23dc95a-d65a-4537-b65e-161954e059fa (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::1a]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: f23dc95a-d65a-4537-b65e-161954e059fa (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:538 sled_address = [fd00:1122:3344:101::1]:12345
11662024-11-18T18:40:01.893ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:60003
11672024-11-18T18:40:01.893ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 55213fec-a100-46b7-9721-e06d9201c8d5, dataset_name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
11682024-11-18T18:40:01.893ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 55213fec-a100-46b7-9721-e06d9201c8d5, dataset_name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: InternalDns }, responder: ... }
11692024-11-18T18:40:01.977ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 07f4fc36-9902-4aeb-a7d9-f99de01ff5c2, dataset_name: DatasetName { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
11702024-11-18T18:40:01.977ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 07f4fc36-9902-4aeb-a7d9-f99de01ff5c2, dataset_name: DatasetName { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, kind: InternalDns }, responder: ... }
11712024-11-18T18:40:02.070ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: b160705f-2e55-4b76-ba79-187b4901f005, dataset_name: DatasetName { pool_name: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
11722024-11-18T18:40:02.070ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: b160705f-2e55-4b76-ba79-187b4901f005, dataset_name: DatasetName { pool_name: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }, kind: InternalDns }, responder: ... }
11732024-11-18T18:40:02.140ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 9d6a1714-4692-4a2d-9475-dc5b067334ad, dataset_name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
11742024-11-18T18:40:02.140ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 9d6a1714-4692-4a2d-9475-dc5b067334ad, dataset_name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: Cockroach }, responder: ... }
11752024-11-18T18:40:02.197ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: e0a33abb-5d16-47b8-a088-6370c29fad2a, dataset_name: DatasetName { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
11762024-11-18T18:40:02.197ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: e0a33abb-5d16-47b8-a088-6370c29fad2a, dataset_name: DatasetName { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, kind: Cockroach }, responder: ... }
11772024-11-18T18:40:02.258ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 509f6459-b46e-474b-8247-9cd9c287fa80, dataset_name: DatasetName { pool_name: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
11782024-11-18T18:40:02.258ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 509f6459-b46e-474b-8247-9cd9c287fa80, dataset_name: DatasetName { pool_name: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }, kind: Cockroach }, responder: ... }
11792024-11-18T18:40:02.329ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: d164bfd1-7ae2-47c5-bc53-968633c77a9d, dataset_name: DatasetName { pool_name: ZpoolName { id: 0199d4fd-4090-4a7c-83e6-040814eb79bb (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
11802024-11-18T18:40:02.330ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: d164bfd1-7ae2-47c5-bc53-968633c77a9d, dataset_name: DatasetName { pool_name: ZpoolName { id: 0199d4fd-4090-4a7c-83e6-040814eb79bb (zpool), kind: External }, kind: Cockroach }, responder: ... }
11812024-11-18T18:40:02.391ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: b9daa5c1-3421-4e6b-889a-4c4b145361da, dataset_name: DatasetName { pool_name: ZpoolName { id: 5840d858-3782-45e1-a6e0-d60d070ad2c3 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
11822024-11-18T18:40:02.391ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: b9daa5c1-3421-4e6b-889a-4c4b145361da, dataset_name: DatasetName { pool_name: ZpoolName { id: 5840d858-3782-45e1-a6e0-d60d070ad2c3 (zpool), kind: External }, kind: Cockroach }, responder: ... }
11832024-11-18T18:40:02.458ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 6767b855-63e1-4a7d-ac38-c4e7eb56747f, dataset_name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: ExternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
11842024-11-18T18:40:02.458ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 6767b855-63e1-4a7d-ac38-c4e7eb56747f, dataset_name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: ExternalDns }, responder: ... }
11852024-11-18T18:40:02.507ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ceea2287-3571-41ea-8f15-b2aeb4a74fd9, dataset_name: DatasetName { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, kind: ExternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
11862024-11-18T18:40:02.508ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: ceea2287-3571-41ea-8f15-b2aeb4a74fd9, dataset_name: DatasetName { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, kind: ExternalDns }, responder: ... }
11872024-11-18T18:40:02.576ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 55ab4bd1-3e97-453c-b90c-176360dea573, dataset_name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: Clickhouse }, responder: ... }) file = sled-storage/src/manager.rs:603
11882024-11-18T18:40:02.576ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 55ab4bd1-3e97-453c-b90c-176360dea573, dataset_name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: Clickhouse }, responder: ... }
11892024-11-18T18:40:02.630ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 396921f6-50a5-459a-86ad-182888d80a34, dataset_name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
11902024-11-18T18:40:02.630ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 396921f6-50a5-459a-86ad-182888d80a34, dataset_name: DatasetName { pool_name: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, kind: Crucible }, responder: ... }
11912024-11-18T18:40:02.685ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: efcbb893-1918-43b9-8315-e9bb90d664e8, dataset_name: DatasetName { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
11922024-11-18T18:40:02.685ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: efcbb893-1918-43b9-8315-e9bb90d664e8, dataset_name: DatasetName { pool_name: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, kind: Crucible }, responder: ... }
11932024-11-18T18:40:02.740ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 6062ae2a-331c-4cb6-a9b5-eb4310b33530, dataset_name: DatasetName { pool_name: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
11942024-11-18T18:40:02.740ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 6062ae2a-331c-4cb6-a9b5-eb4310b33530, dataset_name: DatasetName { pool_name: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }, kind: Crucible }, responder: ... }
11952024-11-18T18:40:02.794ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: bb775b2d-b688-494a-8509-79ecf4eb8f60, dataset_name: DatasetName { pool_name: ZpoolName { id: 0199d4fd-4090-4a7c-83e6-040814eb79bb (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
11962024-11-18T18:40:02.794ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: bb775b2d-b688-494a-8509-79ecf4eb8f60, dataset_name: DatasetName { pool_name: ZpoolName { id: 0199d4fd-4090-4a7c-83e6-040814eb79bb (zpool), kind: External }, kind: Crucible }, responder: ... }
11972024-11-18T18:40:02.847ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: e782f5fc-076e-414c-b87e-6b09d1bdfaeb, dataset_name: DatasetName { pool_name: ZpoolName { id: 5840d858-3782-45e1-a6e0-d60d070ad2c3 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
11982024-11-18T18:40:02.847ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: e782f5fc-076e-414c-b87e-6b09d1bdfaeb, dataset_name: DatasetName { pool_name: ZpoolName { id: 5840d858-3782-45e1-a6e0-d60d070ad2c3 (zpool), kind: External }, kind: Crucible }, responder: ... }
11992024-11-18T18:40:02.908ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 3aad33dc-2f67-4a47-b973-90de986a5f65, dataset_name: DatasetName { pool_name: ZpoolName { id: 131b4d69-dbb0-4397-9284-176e4a3c973d (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
12002024-11-18T18:40:02.908ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 3aad33dc-2f67-4a47-b973-90de986a5f65, dataset_name: DatasetName { pool_name: ZpoolName { id: 131b4d69-dbb0-4397-9284-176e4a3c973d (zpool), kind: External }, kind: Crucible }, responder: ... }
12012024-11-18T18:40:02.978ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 81488621-5898-41d4-a112-d3e574fb2d33, dataset_name: DatasetName { pool_name: ZpoolName { id: 453f7245-5f70-4fe8-b37e-054e56d5c1c5 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
12022024-11-18T18:40:02.978ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 81488621-5898-41d4-a112-d3e574fb2d33, dataset_name: DatasetName { pool_name: ZpoolName { id: 453f7245-5f70-4fe8-b37e-054e56d5c1c5 (zpool), kind: External }, kind: Crucible }, responder: ... }
12032024-11-18T18:40:03.066ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2735d4d6-073c-4860-ae50-e3742cc2cefa, dataset_name: DatasetName { pool_name: ZpoolName { id: b09b6e80-5341-4ded-a520-3f312403e198 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
12042024-11-18T18:40:03.066ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 2735d4d6-073c-4860-ae50-e3742cc2cefa, dataset_name: DatasetName { pool_name: ZpoolName { id: b09b6e80-5341-4ded-a520-3f312403e198 (zpool), kind: External }, kind: Crucible }, responder: ... }
12052024-11-18T18:40:03.146ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 10904f20-d090-41dc-9a54-dab632f01d4a, dataset_name: DatasetName { pool_name: ZpoolName { id: f23dc95a-d65a-4537-b65e-161954e059fa (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
12062024-11-18T18:40:03.146ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 10904f20-d090-41dc-9a54-dab632f01d4a, dataset_name: DatasetName { pool_name: ZpoolName { id: f23dc95a-d65a-4537-b65e-161954e059fa (zpool), kind: External }, kind: Crucible }, responder: ... }
12072024-11-18T18:40:03.225ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12082024-11-18T18:40:03.225ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/9216f0f9-29c9-4d33-8c71-0f237e1331a0/config/omicron-zones.json file = common/src/ledger.rs:177
12092024-11-18T18:40:03.226ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/c23ca4a6-dcd8-4835-a278-26664c492a4d/config/omicron-zones.json file = common/src/ledger.rs:177
12102024-11-18T18:40:03.226ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12112024-11-18T18:40:03.382ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
12122024-11-18T18:40:03.383ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12132024-11-18T18:40:03.410ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
12142024-11-18T18:40:03.410ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12152024-11-18T18:40:03.410ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12162024-11-18T18:40:03.422ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
12172024-11-18T18:40:03.424ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
12182024-11-18T18:40:03.450ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12192024-11-18T18:40:03.450ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12202024-11-18T18:40:03.450ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12212024-11-18T18:40:03.450ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12222024-11-18T18:40:03.656ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
12232024-11-18T18:40:03.657ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12242024-11-18T18:40:03.657ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12252024-11-18T18:40:03.657ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12262024-11-18T18:40:03.657ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12272024-11-18T18:40:03.671ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12282024-11-18T18:40:03.671ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12292024-11-18T18:40:03.671ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12302024-11-18T18:40:03.672ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_efcbb893-1918-43b9-8315-e9bb90d664e8 file = illumos-utils/src/zone.rs:325
12312024-11-18T18:40:03.678ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_81488621-5898-41d4-a112-d3e574fb2d33 file = illumos-utils/src/zone.rs:325
12322024-11-18T18:40:03.683ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_clickhouse_55ab4bd1-3e97-453c-b90c-176360dea573 file = illumos-utils/src/zone.rs:325
12332024-11-18T18:40:03.688ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6 file = illumos-utils/src/zone.rs:325
12342024-11-18T18:40:03.693ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_e782f5fc-076e-414c-b87e-6b09d1bdfaeb file = illumos-utils/src/zone.rs:325
12352024-11-18T18:40:03.698ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_10904f20-d090-41dc-9a54-dab632f01d4a file = illumos-utils/src/zone.rs:325
12362024-11-18T18:40:03.703ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_1fa25dd7-d8f9-49ef-bb20-c6bad413aace file = illumos-utils/src/zone.rs:325
12372024-11-18T18:40:03.850ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_efcbb893-1918-43b9-8315-e9bb90d664e8 file = illumos-utils/src/zone.rs:363
12382024-11-18T18:40:03.855ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_81488621-5898-41d4-a112-d3e574fb2d33 file = illumos-utils/src/zone.rs:363
12392024-11-18T18:40:03.859ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_clickhouse_55ab4bd1-3e97-453c-b90c-176360dea573 file = illumos-utils/src/zone.rs:363
12402024-11-18T18:40:03.865ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6 file = illumos-utils/src/zone.rs:363
12412024-11-18T18:40:03.869ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_e782f5fc-076e-414c-b87e-6b09d1bdfaeb file = illumos-utils/src/zone.rs:363
12422024-11-18T18:40:03.873ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_10904f20-d090-41dc-9a54-dab632f01d4a file = illumos-utils/src/zone.rs:363
12432024-11-18T18:40:03.878ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_1fa25dd7-d8f9-49ef-bb20-c6bad413aace file = illumos-utils/src/zone.rs:363
12442024-11-18T18:40:03.883ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60 file = illumos-utils/src/zone.rs:325
12452024-11-18T18:40:03.889ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_3aad33dc-2f67-4a47-b973-90de986a5f65 file = illumos-utils/src/zone.rs:325
12462024-11-18T18:40:03.895ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_2735d4d6-073c-4860-ae50-e3742cc2cefa file = illumos-utils/src/zone.rs:325
12472024-11-18T18:40:03.901ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_6062ae2a-331c-4cb6-a9b5-eb4310b33530 file = illumos-utils/src/zone.rs:325
12482024-11-18T18:40:03.907ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_023a09b8-2c2b-483c-be8c-540d1a35ce5a file = illumos-utils/src/zone.rs:325
12492024-11-18T18:40:03.914ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46 file = illumos-utils/src/zone.rs:325
12502024-11-18T18:40:03.920ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34 file = illumos-utils/src/zone.rs:325
12512024-11-18T18:40:03.979ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
12522024-11-18T18:40:03.979ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60 file = illumos-utils/src/zone.rs:363
12532024-11-18T18:40:03.988ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_3aad33dc-2f67-4a47-b973-90de986a5f65 file = illumos-utils/src/zone.rs:363
12542024-11-18T18:40:03.996ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_2735d4d6-073c-4860-ae50-e3742cc2cefa file = illumos-utils/src/zone.rs:363
12552024-11-18T18:40:04.007ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_6062ae2a-331c-4cb6-a9b5-eb4310b33530 file = illumos-utils/src/zone.rs:363
12562024-11-18T18:40:04.015ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_023a09b8-2c2b-483c-be8c-540d1a35ce5a file = illumos-utils/src/zone.rs:363
12572024-11-18T18:40:04.023ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46 file = illumos-utils/src/zone.rs:363
12582024-11-18T18:40:04.030ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34 file = illumos-utils/src/zone.rs:363
12592024-11-18T18:40:04.096ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
12602024-11-18T18:40:04.152ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
12612024-11-18T18:40:04.237ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
12622024-11-18T18:40:04.379ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
12632024-11-18T18:40:04.385ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
12642024-11-18T18:40:04.385ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
12652024-11-18T18:40:04.437ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:430 port = Port { inner: PortInner(PortData { name: "opte1", ip: 172.30.2.5, mac: MacAddr6([168, 64, 37, 255, 203, 0]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
12662024-11-18T18:40:04.437ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1264 zone_type = nexus
12672024-11-18T18:40:04.438ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
12682024-11-18T18:40:04.438ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
12692024-11-18T18:40:04.490ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:430 port = Port { inner: PortInner(PortData { name: "opte2", ip: 172.30.1.5, mac: MacAddr6([168, 64, 37, 255, 152, 29]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.1.1 } }) }
12702024-11-18T18:40:04.490ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1264 zone_type = external_dns
12712024-11-18T18:40:04.492ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
12722024-11-18T18:40:04.492ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
12732024-11-18T18:40:04.544ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:430 port = Port { inner: PortInner(PortData { name: "opte3", ip: 172.30.2.6, mac: MacAddr6([168, 64, 37, 255, 222, 37]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
12742024-11-18T18:40:04.544ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1264 zone_type = nexus
12752024-11-18T18:40:04.546ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
12762024-11-18T18:40:04.546ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
12772024-11-18T18:40:04.599ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:430 port = Port { inner: PortInner(PortData { name: "opte4", ip: 172.30.1.6, mac: MacAddr6([168, 64, 37, 255, 170, 19]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.1.1 } }) }
12782024-11-18T18:40:04.600ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1264 zone_type = external_dns
12792024-11-18T18:40:04.601ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
12802024-11-18T18:40:04.601ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
12812024-11-18T18:40:04.655ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:430 port = Port { inner: PortInner(PortData { name: "opte5", ip: 172.30.2.7, mac: MacAddr6([168, 64, 37, 255, 151, 123]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
12822024-11-18T18:40:04.655ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1264 zone_type = nexus
12832024-11-18T18:40:04.669ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.164 file = clients/dpd-client/src/lib.rs:83
12842024-11-18T18:40:04.669ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.164 file = clients/dpd-client/src/lib.rs:90
12852024-11-18T18:40:04.673ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.168 file = clients/dpd-client/src/lib.rs:83
12862024-11-18T18:40:04.673ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.168 file = clients/dpd-client/src/lib.rs:90
12872024-11-18T18:40:04.673ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.166 file = clients/dpd-client/src/lib.rs:83
12882024-11-18T18:40:04.673ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.166 file = clients/dpd-client/src/lib.rs:90
12892024-11-18T18:40:04.675ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.165 file = clients/dpd-client/src/lib.rs:83
12902024-11-18T18:40:04.675ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.165 file = clients/dpd-client/src/lib.rs:90
12912024-11-18T18:40:04.675ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.167 file = clients/dpd-client/src/lib.rs:83
12922024-11-18T18:40:04.676ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.167 file = clients/dpd-client/src/lib.rs:90
12932024-11-18T18:40:04.688ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.168 file = clients/dpd-client/src/lib.rs:118
12942024-11-18T18:40:04.690ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12952024-11-18T18:40:04.739ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.164 file = clients/dpd-client/src/lib.rs:118
12962024-11-18T18:40:04.743ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12972024-11-18T18:40:04.743ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.166 file = clients/dpd-client/src/lib.rs:118
12982024-11-18T18:40:04.749ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.167 file = clients/dpd-client/src/lib.rs:118
12992024-11-18T18:40:04.753ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.165 file = clients/dpd-client/src/lib.rs:118
13002024-11-18T18:40:04.753ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
13012024-11-18T18:40:04.753ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
13022024-11-18T18:40:04.797ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
13032024-11-18T18:40:04.891ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_d7ddf21e-0e1e-4004-b302-ecb81ba84c09 file = illumos-utils/src/zone.rs:325
13042024-11-18T18:40:04.898ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_external_dns_6767b855-63e1-4a7d-ac38-c4e7eb56747f file = illumos-utils/src/zone.rs:325
13052024-11-18T18:40:04.906ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1 file = illumos-utils/src/zone.rs:325
13062024-11-18T18:40:04.914ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d file = illumos-utils/src/zone.rs:325
13072024-11-18T18:40:04.922ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_external_dns_ceea2287-3571-41ea-8f15-b2aeb4a74fd9 file = illumos-utils/src/zone.rs:325
13082024-11-18T18:40:04.930ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_d7ddf21e-0e1e-4004-b302-ecb81ba84c09 file = illumos-utils/src/zone.rs:363
13092024-11-18T18:40:04.937ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_external_dns_6767b855-63e1-4a7d-ac38-c4e7eb56747f file = illumos-utils/src/zone.rs:363
13102024-11-18T18:40:04.945ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1 file = illumos-utils/src/zone.rs:363
13112024-11-18T18:40:04.953ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d file = illumos-utils/src/zone.rs:363
13122024-11-18T18:40:04.960ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_external_dns_ceea2287-3571-41ea-8f15-b2aeb4a74fd9 file = illumos-utils/src/zone.rs:363
13132024-11-18T18:40:13.390ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService22'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::11'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::11'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13142024-11-18T18:40:13.391ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46
13152024-11-18T18:40:13.572ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_81488621-5898-41d4-a112-d3e574fb2d33: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService11'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::18'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_453f7245-5f70-4fe8-b37e-054e56d5c1c5/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::18'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='6f677d71-a360-4af4-ba83-0632249e7d70'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13162024-11-18T18:40:13.573ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_e782f5fc-076e-414c-b87e-6b09d1bdfaeb: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService14'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::16'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_5840d858-3782-45e1-a6e0-d60d070ad2c3/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::16'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='00817188-caa4-4fd6-aec8-009e9c727fd9'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13172024-11-18T18:40:13.574ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_10904f20-d090-41dc-9a54-dab632f01d4a: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService15'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::1a'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_f23dc95a-d65a-4537-b65e-161954e059fa/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::1a'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='83ae08ee-f24b-4945-806c-64568ee31d6d'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13182024-11-18T18:40:13.574ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService17'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::15'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_0199d4fd-4090-4a7c-83e6-040814eb79bb/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::15'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='83447489-4534-4dc7-8ea3-be316fe7ff06'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13192024-11-18T18:40:13.574ZINFOSledAgent (ServiceManager): Booting oxz_crucible_81488621-5898-41d4-a112-d3e574fb2d33 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_81488621-5898-41d4-a112-d3e574fb2d33
13202024-11-18T18:40:13.585ZINFOSledAgent (ServiceManager): Booting oxz_crucible_e782f5fc-076e-414c-b87e-6b09d1bdfaeb zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_e782f5fc-076e-414c-b87e-6b09d1bdfaeb
13212024-11-18T18:40:13.594ZINFOSledAgent (ServiceManager): Booting oxz_crucible_10904f20-d090-41dc-9a54-dab632f01d4a zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_10904f20-d090-41dc-9a54-dab632f01d4a
13222024-11-18T18:40:13.601ZINFOSledAgent (ServiceManager): Booting oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60
13232024-11-18T18:40:13.757ZINFOSledAgent (ServiceManager): Profile for oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService13'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::d'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/oximeter"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="id" value='8716f96b-ee60-45b3-a886-a68fcf5530d6'/> <propval type="astring" name="address" value='[fd00:1122:3344:101::d]:12223'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13242024-11-18T18:40:13.757ZINFOSledAgent (ServiceManager): Booting oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6
13252024-11-18T18:40:13.977ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_023a09b8-2c2b-483c-be8c-540d1a35ce5a: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService21'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::10'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::10'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13262024-11-18T18:40:14.057ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_023a09b8-2c2b-483c-be8c-540d1a35ce5a zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_023a09b8-2c2b-483c-be8c-540d1a35ce5a
13272024-11-18T18:40:14.854ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_3aad33dc-2f67-4a47-b973-90de986a5f65: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService18'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::17'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_131b4d69-dbb0-4397-9284-176e4a3c973d/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::17'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='899e9c12-76ce-4660-9dc0-0191ecc0ea19'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13282024-11-18T18:40:14.856ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_1fa25dd7-d8f9-49ef-bb20-c6bad413aace: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService16'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::f'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::f'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13292024-11-18T18:40:14.856ZINFOSledAgent (ServiceManager): Booting oxz_crucible_3aad33dc-2f67-4a47-b973-90de986a5f65 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_3aad33dc-2f67-4a47-b973-90de986a5f65
13302024-11-18T18:40:14.871ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_1fa25dd7-d8f9-49ef-bb20-c6bad413aace zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_1fa25dd7-d8f9-49ef-bb20-c6bad413aace
13312024-11-18T18:40:14.884ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_2735d4d6-073c-4860-ae50-e3742cc2cefa: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService19'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::19'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_b09b6e80-5341-4ded-a520-3f312403e198/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::19'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='ec260073-e9b0-45d0-9fc0-2ed3cad2fbeb'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13322024-11-18T18:40:14.886ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_6062ae2a-331c-4cb6-a9b5-eb4310b33530: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService20'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::14'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_df4faf1b-f747-4cda-8481-deaa8523a533/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::14'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='da12475d-e530-4208-a61f-69a789e8d2a9'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13332024-11-18T18:40:14.888ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService23'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::12'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_4be98a0c-ab4c-4309-948b-04d776603876/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::12'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='e1bcfb03-f946-4f34-8096-fe20dc8a1221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13342024-11-18T18:40:14.890ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_efcbb893-1918-43b9-8315-e9bb90d664e8: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService10'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::13'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_4f35fd7e-936a-4e09-946b-7cb174580897/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::13'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='ee619136-25bc-408f-8a25-0b18eb6a4ad2'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13352024-11-18T18:40:14.890ZINFOSledAgent (ServiceManager): Booting oxz_crucible_2735d4d6-073c-4860-ae50-e3742cc2cefa zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_2735d4d6-073c-4860-ae50-e3742cc2cefa
13362024-11-18T18:40:24.943ZINFOSledAgent (ServiceManager): Booting oxz_crucible_6062ae2a-331c-4cb6-a9b5-eb4310b33530 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_6062ae2a-331c-4cb6-a9b5-eb4310b33530
13372024-11-18T18:40:25.158ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_023a09b8-2c2b-483c-be8c-540d1a35ce5a") failed: Property not found. retry in 40.217772ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_023a09b8-2c2b-483c-be8c-540d1a35ce5a
13382024-11-18T18:40:25.498ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46") failed: Property not found. retry in 68.680214ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46
13392024-11-18T18:40:25.498ZINFOSledAgent (ServiceManager): Booting oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34
13402024-11-18T18:40:25.593ZINFOSledAgent (ServiceManager): Booting oxz_crucible_efcbb893-1918-43b9-8315-e9bb90d664e8 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_efcbb893-1918-43b9-8315-e9bb90d664e8
13412024-11-18T18:40:25.772ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_81488621-5898-41d4-a112-d3e574fb2d33") failed: Property not found. retry in 51.839241ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_81488621-5898-41d4-a112-d3e574fb2d33
13422024-11-18T18:40:25.772ZINFOSledAgent (ServiceManager): Profile for oxz_external_dns_ceea2287-3571-41ea-8f15-b2aeb4a74fd9: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService28'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::9'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte4'/> <propval type="astring" name="gateway" value='172.30.1.1'/> <propval type="astring" name="ip" value='172.30.1.6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/external_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::9]:5353'/> <propval type="astring" name="dns_address" value='172.30.1.6:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13432024-11-18T18:40:25.818ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6") failed: Property not found. retry in 45.287337ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6
13442024-11-18T18:40:25.867ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e782f5fc-076e-414c-b87e-6b09d1bdfaeb") failed: Property not found. retry in 34.228056ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e782f5fc-076e-414c-b87e-6b09d1bdfaeb
13452024-11-18T18:40:25.965ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_10904f20-d090-41dc-9a54-dab632f01d4a") failed: Property not found. retry in 40.245016ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_10904f20-d090-41dc-9a54-dab632f01d4a
13462024-11-18T18:40:26.026ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1fa25dd7-d8f9-49ef-bb20-c6bad413aace") failed: Property not found. retry in 65.385519ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1fa25dd7-d8f9-49ef-bb20-c6bad413aace
13472024-11-18T18:40:26.088ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60") failed: Property not found. retry in 64.09421ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60
13482024-11-18T18:40:26.137ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 38.787511269s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
13492024-11-18T18:40:26.182ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aad33dc-2f67-4a47-b973-90de986a5f65") failed: Property not found. retry in 28.551058ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aad33dc-2f67-4a47-b973-90de986a5f65
13502024-11-18T18:40:26.189ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2735d4d6-073c-4860-ae50-e3742cc2cefa") failed: Property not found. retry in 45.727768ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2735d4d6-073c-4860-ae50-e3742cc2cefa
13512024-11-18T18:40:26.198ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6062ae2a-331c-4cb6-a9b5-eb4310b33530") failed: Property not found. retry in 34.659246ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6062ae2a-331c-4cb6-a9b5-eb4310b33530
13522024-11-18T18:40:26.198ZINFOSledAgent (ServiceManager): Profile for oxz_external_dns_6767b855-63e1-4a7d-ac38-c4e7eb56747f: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService25'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::8'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte2'/> <propval type="astring" name="gateway" value='172.30.1.1'/> <propval type="astring" name="ip" value='172.30.1.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/external_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::8]:5353'/> <propval type="astring" name="dns_address" value='172.30.1.5:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13532024-11-18T18:40:26.207ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_023a09b8-2c2b-483c-be8c-540d1a35ce5a") failed: Property not found. retry in 143.776057ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_023a09b8-2c2b-483c-be8c-540d1a35ce5a
13542024-11-18T18:40:26.216ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46") failed: Property not found. retry in 133.259028ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46
13552024-11-18T18:40:26.225ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34") failed: Property not found. retry in 36.044717ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34
13562024-11-18T18:40:26.235ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_efcbb893-1918-43b9-8315-e9bb90d664e8") failed: Property not found. retry in 33.974146ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_efcbb893-1918-43b9-8315-e9bb90d664e8
13572024-11-18T18:40:26.244ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_81488621-5898-41d4-a112-d3e574fb2d33") failed: Property not found. retry in 65.724746ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_81488621-5898-41d4-a112-d3e574fb2d33
13582024-11-18T18:40:26.244ZINFOSledAgent (ServiceManager): Booting oxz_external_dns_ceea2287-3571-41ea-8f15-b2aeb4a74fd9 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_external_dns_ceea2287-3571-41ea-8f15-b2aeb4a74fd9
13592024-11-18T18:40:26.299ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6") failed: Property not found. retry in 137.564167ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6
13602024-11-18T18:40:26.308ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e782f5fc-076e-414c-b87e-6b09d1bdfaeb") failed: Property not found. retry in 92.840673ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e782f5fc-076e-414c-b87e-6b09d1bdfaeb
13612024-11-18T18:40:26.317ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_10904f20-d090-41dc-9a54-dab632f01d4a") failed: Property not found. retry in 75.89409ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_10904f20-d090-41dc-9a54-dab632f01d4a
13622024-11-18T18:40:26.325ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1fa25dd7-d8f9-49ef-bb20-c6bad413aace") failed: Property not found. retry in 107.756354ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1fa25dd7-d8f9-49ef-bb20-c6bad413aace
13632024-11-18T18:40:26.333ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60") failed: Property not found. retry in 108.574855ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60
13642024-11-18T18:40:26.341ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aad33dc-2f67-4a47-b973-90de986a5f65") failed: Property not found. retry in 53.654484ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aad33dc-2f67-4a47-b973-90de986a5f65
13652024-11-18T18:40:26.350ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2735d4d6-073c-4860-ae50-e3742cc2cefa") failed: Property not found. retry in 113.674825ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2735d4d6-073c-4860-ae50-e3742cc2cefa
13662024-11-18T18:40:26.362ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6062ae2a-331c-4cb6-a9b5-eb4310b33530") failed: Property not found. retry in 85.442154ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6062ae2a-331c-4cb6-a9b5-eb4310b33530
13672024-11-18T18:40:26.362ZINFOSledAgent (ServiceManager): Booting oxz_external_dns_6767b855-63e1-4a7d-ac38-c4e7eb56747f zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_external_dns_6767b855-63e1-4a7d-ac38-c4e7eb56747f
13682024-11-18T18:40:26.381ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_023a09b8-2c2b-483c-be8c-540d1a35ce5a") failed: Property not found. retry in 194.46603ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_023a09b8-2c2b-483c-be8c-540d1a35ce5a
13692024-11-18T18:40:26.533ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46") failed: Property not found. retry in 281.028991ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46
13702024-11-18T18:40:26.558ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34") failed: Property not found. retry in 103.51825ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34
13712024-11-18T18:40:26.575ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_efcbb893-1918-43b9-8315-e9bb90d664e8") failed: Property not found. retry in 69.203543ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_efcbb893-1918-43b9-8315-e9bb90d664e8
13722024-11-18T18:40:26.594ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_81488621-5898-41d4-a112-d3e574fb2d33") failed: Property not found. retry in 216.433933ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_81488621-5898-41d4-a112-d3e574fb2d33
13732024-11-18T18:40:26.607ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_ceea2287-3571-41ea-8f15-b2aeb4a74fd9") failed: Property not found. retry in 68.333605ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_ceea2287-3571-41ea-8f15-b2aeb4a74fd9
13742024-11-18T18:40:26.646ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6") failed: Property not found. retry in 286.05459ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6
13752024-11-18T18:40:26.670ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e782f5fc-076e-414c-b87e-6b09d1bdfaeb") failed: Property not found. retry in 158.340998ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e782f5fc-076e-414c-b87e-6b09d1bdfaeb
13762024-11-18T18:40:26.679ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_10904f20-d090-41dc-9a54-dab632f01d4a") failed: Property not found. retry in 279.723425ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_10904f20-d090-41dc-9a54-dab632f01d4a
13772024-11-18T18:40:26.707ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1fa25dd7-d8f9-49ef-bb20-c6bad413aace") failed: Property not found. retry in 236.672126ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1fa25dd7-d8f9-49ef-bb20-c6bad413aace
13782024-11-18T18:40:26.726ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60") failed: Property not found. retry in 146.011584ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60
13792024-11-18T18:40:26.738ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aad33dc-2f67-4a47-b973-90de986a5f65") failed: Property not found. retry in 198.27372ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aad33dc-2f67-4a47-b973-90de986a5f65
13802024-11-18T18:40:26.752ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2735d4d6-073c-4860-ae50-e3742cc2cefa") failed: Property not found. retry in 185.709277ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2735d4d6-073c-4860-ae50-e3742cc2cefa
13812024-11-18T18:40:26.763ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6062ae2a-331c-4cb6-a9b5-eb4310b33530") failed: Property not found. retry in 288.389458ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6062ae2a-331c-4cb6-a9b5-eb4310b33530
13822024-11-18T18:40:26.775ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_023a09b8-2c2b-483c-be8c-540d1a35ce5a") failed: Property not found. retry in 465.740894ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_023a09b8-2c2b-483c-be8c-540d1a35ce5a
13832024-11-18T18:40:26.785ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34") failed: Property not found. retry in 120.589925ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34
13842024-11-18T18:40:26.795ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_efcbb893-1918-43b9-8315-e9bb90d664e8") failed: Property not found. retry in 285.466384ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_efcbb893-1918-43b9-8315-e9bb90d664e8
13852024-11-18T18:40:26.827ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_ceea2287-3571-41ea-8f15-b2aeb4a74fd9") failed: Property not found. retry in 68.888421ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_ceea2287-3571-41ea-8f15-b2aeb4a74fd9
13862024-11-18T18:40:26.827ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_d7ddf21e-0e1e-4004-b302-ecb81ba84c09: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService24'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::c'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte5'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.7'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13872024-11-18T18:40:26.835ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e782f5fc-076e-414c-b87e-6b09d1bdfaeb") failed: Property not found. retry in 490.326287ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e782f5fc-076e-414c-b87e-6b09d1bdfaeb
13882024-11-18T18:40:26.844ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_6767b855-63e1-4a7d-ac38-c4e7eb56747f") failed: Property not found. retry in 45.161429ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_6767b855-63e1-4a7d-ac38-c4e7eb56747f
13892024-11-18T18:40:26.854ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46") failed: Property not found. retry in 488.028304ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46
13902024-11-18T18:40:26.854ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService26'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::a'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte1'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13912024-11-18T18:40:26.865ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_81488621-5898-41d4-a112-d3e574fb2d33") failed: Property not found. retry in 408.774808ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_81488621-5898-41d4-a112-d3e574fb2d33
13922024-11-18T18:40:26.865ZINFOSledAgent (ServiceManager): Booting oxz_nexus_d7ddf21e-0e1e-4004-b302-ecb81ba84c09 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_d7ddf21e-0e1e-4004-b302-ecb81ba84c09
13932024-11-18T18:40:26.884ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60") failed: Property not found. retry in 453.387431ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60
13942024-11-18T18:40:26.884ZINFOSledAgent (ServiceManager): Booting oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1
13952024-11-18T18:40:26.890ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService27'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::b'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte3'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13962024-11-18T18:40:26.890ZINFOSledAgent (ServiceManager): Booting oxz_nexus_88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d
13972024-11-18T18:40:26.910ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_ceea2287-3571-41ea-8f15-b2aeb4a74fd9") failed: Property not found. retry in 143.37277ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_ceea2287-3571-41ea-8f15-b2aeb4a74fd9
13982024-11-18T18:40:26.918ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_6767b855-63e1-4a7d-ac38-c4e7eb56747f") failed: Property not found. retry in 101.014524ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_6767b855-63e1-4a7d-ac38-c4e7eb56747f
13992024-11-18T18:40:26.930ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34") failed: Property not found. retry in 541.940022ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34
14002024-11-18T18:40:26.940ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6") failed: Property not found. retry in 352.564163ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6
14012024-11-18T18:40:26.953ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aad33dc-2f67-4a47-b973-90de986a5f65") failed: Property not found. retry in 589.005571ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aad33dc-2f67-4a47-b973-90de986a5f65
14022024-11-18T18:40:26.964ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2735d4d6-073c-4860-ae50-e3742cc2cefa") failed: Property not found. retry in 409.343764ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2735d4d6-073c-4860-ae50-e3742cc2cefa
14032024-11-18T18:40:26.974ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_10904f20-d090-41dc-9a54-dab632f01d4a") failed: Property not found. retry in 392.985381ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_10904f20-d090-41dc-9a54-dab632f01d4a
14042024-11-18T18:40:26.984ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1fa25dd7-d8f9-49ef-bb20-c6bad413aace") failed: Property not found. retry in 415.99286ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1fa25dd7-d8f9-49ef-bb20-c6bad413aace
14052024-11-18T18:40:27.032ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_6767b855-63e1-4a7d-ac38-c4e7eb56747f") failed: Property not found. retry in 285.396688ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_6767b855-63e1-4a7d-ac38-c4e7eb56747f
14062024-11-18T18:40:27.063ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6062ae2a-331c-4cb6-a9b5-eb4310b33530") failed: Property not found. retry in 387.115503ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6062ae2a-331c-4cb6-a9b5-eb4310b33530
14072024-11-18T18:40:27.072ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1") failed: Property not found. retry in 60.771162ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1
14082024-11-18T18:40:27.080ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_ceea2287-3571-41ea-8f15-b2aeb4a74fd9") failed: Property not found. retry in 385.194587ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_ceea2287-3571-41ea-8f15-b2aeb4a74fd9
14092024-11-18T18:40:27.087ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d") failed: Property not found. retry in 47.060696ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d
14102024-11-18T18:40:27.094ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d7ddf21e-0e1e-4004-b302-ecb81ba84c09") failed: Property not found. retry in 70.698378ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d7ddf21e-0e1e-4004-b302-ecb81ba84c09
14112024-11-18T18:40:27.102ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_efcbb893-1918-43b9-8315-e9bb90d664e8") failed: Property not found. retry in 462.014972ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_efcbb893-1918-43b9-8315-e9bb90d664e8
14122024-11-18T18:40:27.143ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1") failed: Property not found. retry in 72.007624ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1
14132024-11-18T18:40:27.150ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d") failed: Property not found. retry in 135.143062ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d
14142024-11-18T18:40:27.174ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d7ddf21e-0e1e-4004-b302-ecb81ba84c09") failed: Property not found. retry in 145.031462ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d7ddf21e-0e1e-4004-b302-ecb81ba84c09
14152024-11-18T18:40:27.223ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1") failed: Property not found. retry in 159.749683ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1
14162024-11-18T18:40:27.251ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_023a09b8-2c2b-483c-be8c-540d1a35ce5a") failed: Property not found. retry in 1.105883763s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_023a09b8-2c2b-483c-be8c-540d1a35ce5a
14172024-11-18T18:40:27.285ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_81488621-5898-41d4-a112-d3e574fb2d33") failed: Property not found. retry in 686.753412ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_81488621-5898-41d4-a112-d3e574fb2d33
14182024-11-18T18:40:27.297ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d") failed: Property not found. retry in 299.419326ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d
14192024-11-18T18:40:27.308ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6") failed: Property not found. retry in 1.153452979s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6
14202024-11-18T18:40:27.327ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_6767b855-63e1-4a7d-ac38-c4e7eb56747f") failed: Property not found. retry in 413.604649ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_6767b855-63e1-4a7d-ac38-c4e7eb56747f
14212024-11-18T18:40:27.339ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d7ddf21e-0e1e-4004-b302-ecb81ba84c09") failed: Property not found. retry in 197.089061ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d7ddf21e-0e1e-4004-b302-ecb81ba84c09
14222024-11-18T18:40:27.349ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e782f5fc-076e-414c-b87e-6b09d1bdfaeb") failed: Property not found. retry in 810.65944ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e782f5fc-076e-414c-b87e-6b09d1bdfaeb
14232024-11-18T18:40:27.358ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60") failed: Property not found. retry in 458.813962ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60
14242024-11-18T18:40:27.368ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46") failed: Property not found. retry in 1.011971654s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46
14252024-11-18T18:40:27.378ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_10904f20-d090-41dc-9a54-dab632f01d4a") failed: Property not found. retry in 893.876423ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_10904f20-d090-41dc-9a54-dab632f01d4a
14262024-11-18T18:40:27.391ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2735d4d6-073c-4860-ae50-e3742cc2cefa") failed: Property not found. retry in 785.671879ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2735d4d6-073c-4860-ae50-e3742cc2cefa
14272024-11-18T18:40:27.401ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1") failed: Property not found. retry in 248.103774ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1
14282024-11-18T18:40:27.412ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1fa25dd7-d8f9-49ef-bb20-c6bad413aace") failed: Property not found. retry in 665.739144ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1fa25dd7-d8f9-49ef-bb20-c6bad413aace
14292024-11-18T18:40:27.460ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6062ae2a-331c-4cb6-a9b5-eb4310b33530") failed: Property not found. retry in 545.133414ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6062ae2a-331c-4cb6-a9b5-eb4310b33530
14302024-11-18T18:40:27.476ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_ceea2287-3571-41ea-8f15-b2aeb4a74fd9") failed: Property not found. retry in 497.704556ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_ceea2287-3571-41ea-8f15-b2aeb4a74fd9
14312024-11-18T18:40:27.484ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34") failed: Property not found. retry in 454.961014ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34
14322024-11-18T18:40:27.547ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d7ddf21e-0e1e-4004-b302-ecb81ba84c09") failed: Property not found. retry in 355.057487ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d7ddf21e-0e1e-4004-b302-ecb81ba84c09
14332024-11-18T18:40:27.556ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aad33dc-2f67-4a47-b973-90de986a5f65") failed: Property not found. retry in 1.105656492s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aad33dc-2f67-4a47-b973-90de986a5f65
14342024-11-18T18:40:27.574ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_efcbb893-1918-43b9-8315-e9bb90d664e8") failed: Property not found. retry in 1.068327111s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_efcbb893-1918-43b9-8315-e9bb90d664e8
14352024-11-18T18:40:27.606ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d") failed: Property not found. retry in 346.493233ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d
14362024-11-18T18:40:27.662ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1") failed: Property not found. retry in 917.726062ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1
14372024-11-18T18:40:27.753ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_6767b855-63e1-4a7d-ac38-c4e7eb56747f") failed: Property not found. retry in 1.117409469s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_6767b855-63e1-4a7d-ac38-c4e7eb56747f
14382024-11-18T18:40:27.828ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60") failed: Property not found. retry in 1.240416777s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60
14392024-11-18T18:40:27.912ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d7ddf21e-0e1e-4004-b302-ecb81ba84c09") failed: Property not found. retry in 722.888257ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d7ddf21e-0e1e-4004-b302-ecb81ba84c09
14402024-11-18T18:40:27.950ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34") failed: Property not found. retry in 790.653115ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34
14412024-11-18T18:40:27.963ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d") failed: Property not found. retry in 620.221283ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d
14422024-11-18T18:40:27.981ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_81488621-5898-41d4-a112-d3e574fb2d33") failed: Property not found. retry in 1.434710037s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_81488621-5898-41d4-a112-d3e574fb2d33
14432024-11-18T18:40:27.990ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_ceea2287-3571-41ea-8f15-b2aeb4a74fd9") failed: Property not found. retry in 1.057788367s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_ceea2287-3571-41ea-8f15-b2aeb4a74fd9
14442024-11-18T18:40:28.015ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6062ae2a-331c-4cb6-a9b5-eb4310b33530") failed: Property not found. retry in 1.480865762s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6062ae2a-331c-4cb6-a9b5-eb4310b33530
14452024-11-18T18:40:28.087ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1fa25dd7-d8f9-49ef-bb20-c6bad413aace") failed: Property not found. retry in 569.234218ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1fa25dd7-d8f9-49ef-bb20-c6bad413aace
14462024-11-18T18:40:28.169ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e782f5fc-076e-414c-b87e-6b09d1bdfaeb") failed: Property not found. retry in 690.013927ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e782f5fc-076e-414c-b87e-6b09d1bdfaeb
14472024-11-18T18:40:28.188ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2735d4d6-073c-4860-ae50-e3742cc2cefa") failed: Property not found. retry in 1.354281978s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2735d4d6-073c-4860-ae50-e3742cc2cefa
14482024-11-18T18:40:28.280ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_10904f20-d090-41dc-9a54-dab632f01d4a") failed: Property not found. retry in 1.223888164s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_10904f20-d090-41dc-9a54-dab632f01d4a
14492024-11-18T18:40:28.366ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_023a09b8-2c2b-483c-be8c-540d1a35ce5a") failed: Property not found. retry in 1.336937544s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_023a09b8-2c2b-483c-be8c-540d1a35ce5a
14502024-11-18T18:40:28.388ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46") failed: Property not found. retry in 550.684806ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46
14512024-11-18T18:40:28.483ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6") failed: Property not found. retry in 822.216165ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6
14522024-11-18T18:40:28.592ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1") failed: Property not found. retry in 864.302928ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1
14532024-11-18T18:40:28.601ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d") failed: Property not found. retry in 1.355233262s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d
14542024-11-18T18:40:28.645ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d7ddf21e-0e1e-4004-b302-ecb81ba84c09") failed: Property not found. retry in 745.529689ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d7ddf21e-0e1e-4004-b302-ecb81ba84c09
14552024-11-18T18:40:28.655ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_efcbb893-1918-43b9-8315-e9bb90d664e8") failed: Property not found. retry in 1.118416698s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_efcbb893-1918-43b9-8315-e9bb90d664e8
14562024-11-18T18:40:28.666ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1fa25dd7-d8f9-49ef-bb20-c6bad413aace") failed: Property not found. retry in 1.451452975s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1fa25dd7-d8f9-49ef-bb20-c6bad413aace
14572024-11-18T18:40:28.674ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aad33dc-2f67-4a47-b973-90de986a5f65") failed: Property not found. retry in 1.063796294s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aad33dc-2f67-4a47-b973-90de986a5f65
14582024-11-18T18:40:28.749ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34") failed: Property not found. retry in 1.328920413s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34
14592024-11-18T18:40:28.871ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e782f5fc-076e-414c-b87e-6b09d1bdfaeb") failed: Property not found. retry in 813.198667ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e782f5fc-076e-414c-b87e-6b09d1bdfaeb
14602024-11-18T18:40:28.881ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_6767b855-63e1-4a7d-ac38-c4e7eb56747f") failed: Property not found. retry in 851.708473ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_6767b855-63e1-4a7d-ac38-c4e7eb56747f
14612024-11-18T18:40:28.953ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46") failed: Property not found. retry in 1.167722693s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46
14622024-11-18T18:40:28.955ZINFOSledAgent (ServiceManager): Profile for oxz_clickhouse_55ab4bd1-3e97-453c-b90c-176360dea573: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService12'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::e'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/clickhouse"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::e'/> <propval type="astring" name="listen_port" value='8123'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/clickhouse-admin-single"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::e]:8888'/> <propval type="astring" name="ch_address" value='[fd00:1122:3344:101::e]:8123'/> <propval type="astring" name="ch_binary" value='/opt/oxide/clickhouse/clickhouse'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14632024-11-18T18:40:28.955ZINFOSledAgent (ServiceManager): Booting oxz_clickhouse_55ab4bd1-3e97-453c-b90c-176360dea573 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_clickhouse_55ab4bd1-3e97-453c-b90c-176360dea573
14642024-11-18T18:40:29.060ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_ceea2287-3571-41ea-8f15-b2aeb4a74fd9") failed: Property not found. retry in 1.129408199s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_ceea2287-3571-41ea-8f15-b2aeb4a74fd9
14652024-11-18T18:40:29.081ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60") failed: Property not found. retry in 709.419053ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60
14662024-11-18T18:40:29.219ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_55ab4bd1-3e97-453c-b90c-176360dea573") failed: Property not found. retry in 34.589234ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_55ab4bd1-3e97-453c-b90c-176360dea573
14672024-11-18T18:40:29.264ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_55ab4bd1-3e97-453c-b90c-176360dea573") failed: Property not found. retry in 88.551491ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_55ab4bd1-3e97-453c-b90c-176360dea573
14682024-11-18T18:40:29.317ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6") failed: Property not found. retry in 606.223147ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6
14692024-11-18T18:40:29.364ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_55ab4bd1-3e97-453c-b90c-176360dea573") failed: Property not found. retry in 254.889278ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_55ab4bd1-3e97-453c-b90c-176360dea573
14702024-11-18T18:40:29.409ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d7ddf21e-0e1e-4004-b302-ecb81ba84c09") failed: Property not found. retry in 1.397392244s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d7ddf21e-0e1e-4004-b302-ecb81ba84c09
14712024-11-18T18:40:29.428ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_81488621-5898-41d4-a112-d3e574fb2d33") failed: Property not found. retry in 535.59653ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_81488621-5898-41d4-a112-d3e574fb2d33
14722024-11-18T18:40:29.469ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1") failed: Property not found. retry in 603.587563ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1
14732024-11-18T18:40:29.509ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6062ae2a-331c-4cb6-a9b5-eb4310b33530") failed: Property not found. retry in 534.778846ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6062ae2a-331c-4cb6-a9b5-eb4310b33530
14742024-11-18T18:40:29.520ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_10904f20-d090-41dc-9a54-dab632f01d4a") failed: Property not found. retry in 1.171180636s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_10904f20-d090-41dc-9a54-dab632f01d4a
14752024-11-18T18:40:29.554ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2735d4d6-073c-4860-ae50-e3742cc2cefa") failed: Property not found. retry in 1.384158021s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2735d4d6-073c-4860-ae50-e3742cc2cefa
14762024-11-18T18:40:29.634ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_55ab4bd1-3e97-453c-b90c-176360dea573") failed: Property not found. retry in 539.063538ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_55ab4bd1-3e97-453c-b90c-176360dea573
14772024-11-18T18:40:29.698ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e782f5fc-076e-414c-b87e-6b09d1bdfaeb") failed: Property not found. retry in 603.987025ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e782f5fc-076e-414c-b87e-6b09d1bdfaeb
14782024-11-18T18:40:29.714ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_023a09b8-2c2b-483c-be8c-540d1a35ce5a") failed: Property not found. retry in 505.928347ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_023a09b8-2c2b-483c-be8c-540d1a35ce5a
14792024-11-18T18:40:29.745ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_6767b855-63e1-4a7d-ac38-c4e7eb56747f") failed: Property not found. retry in 647.296007ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_6767b855-63e1-4a7d-ac38-c4e7eb56747f
14802024-11-18T18:40:29.758ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aad33dc-2f67-4a47-b973-90de986a5f65") failed: Property not found. retry in 694.266222ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aad33dc-2f67-4a47-b973-90de986a5f65
14812024-11-18T18:40:29.785ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_efcbb893-1918-43b9-8315-e9bb90d664e8") failed: Property not found. retry in 1.417515305s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_efcbb893-1918-43b9-8315-e9bb90d664e8
14822024-11-18T18:40:29.802ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60") failed: Property not found. retry in 766.783509ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60
14832024-11-18T18:40:29.935ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6") failed: Property not found. retry in 1.168510015s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6
14842024-11-18T18:40:29.968ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d") failed: Property not found. retry in 1.410771966s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d
14852024-11-18T18:40:29.981ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_81488621-5898-41d4-a112-d3e574fb2d33") failed: Property not found. retry in 1.05056707s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_81488621-5898-41d4-a112-d3e574fb2d33
14862024-11-18T18:40:30.056ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6062ae2a-331c-4cb6-a9b5-eb4310b33530") failed: Property not found. retry in 831.326154ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6062ae2a-331c-4cb6-a9b5-eb4310b33530
14872024-11-18T18:40:30.086ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1") failed: Property not found. retry in 1.210066182s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1
14882024-11-18T18:40:30.100ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34") failed: Property not found. retry in 979.967988ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34
14892024-11-18T18:40:30.130ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1fa25dd7-d8f9-49ef-bb20-c6bad413aace") failed: Property not found. retry in 1.400113132s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1fa25dd7-d8f9-49ef-bb20-c6bad413aace
14902024-11-18T18:40:30.141ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46") failed: Property not found. retry in 658.937182ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46
14912024-11-18T18:40:30.186ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_55ab4bd1-3e97-453c-b90c-176360dea573") failed: Property not found. retry in 1.156567852s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_55ab4bd1-3e97-453c-b90c-176360dea573
14922024-11-18T18:40:30.201ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_ceea2287-3571-41ea-8f15-b2aeb4a74fd9") failed: Property not found. retry in 1.420310449s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_ceea2287-3571-41ea-8f15-b2aeb4a74fd9
14932024-11-18T18:40:30.231ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_023a09b8-2c2b-483c-be8c-540d1a35ce5a") failed: Property not found. retry in 1.099363362s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_023a09b8-2c2b-483c-be8c-540d1a35ce5a
14942024-11-18T18:40:30.316ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e782f5fc-076e-414c-b87e-6b09d1bdfaeb") failed: Property not found. retry in 678.679044ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e782f5fc-076e-414c-b87e-6b09d1bdfaeb
14952024-11-18T18:40:30.404ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_6767b855-63e1-4a7d-ac38-c4e7eb56747f") failed: Property not found. retry in 1.446737593s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_6767b855-63e1-4a7d-ac38-c4e7eb56747f
14962024-11-18T18:40:30.464ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aad33dc-2f67-4a47-b973-90de986a5f65") failed: Property not found. retry in 1.410394761s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aad33dc-2f67-4a47-b973-90de986a5f65
14972024-11-18T18:40:30.583ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60") failed: Property not found. retry in 646.01358ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60
14982024-11-18T18:40:30.703ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_10904f20-d090-41dc-9a54-dab632f01d4a") failed: Property not found. retry in 1.207795166s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_10904f20-d090-41dc-9a54-dab632f01d4a
14992024-11-18T18:40:30.812ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46") failed: Property not found. retry in 875.398435ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46
15002024-11-18T18:40:30.823ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d7ddf21e-0e1e-4004-b302-ecb81ba84c09") failed: Property not found. retry in 857.505787ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d7ddf21e-0e1e-4004-b302-ecb81ba84c09
15012024-11-18T18:40:30.900ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6062ae2a-331c-4cb6-a9b5-eb4310b33530") failed: Property not found. retry in 1.104128515s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6062ae2a-331c-4cb6-a9b5-eb4310b33530
15022024-11-18T18:40:30.951ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2735d4d6-073c-4860-ae50-e3742cc2cefa") failed: Property not found. retry in 1.227147663s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2735d4d6-073c-4860-ae50-e3742cc2cefa
15032024-11-18T18:40:31.006ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e782f5fc-076e-414c-b87e-6b09d1bdfaeb") failed: Property not found. retry in 763.79887ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e782f5fc-076e-414c-b87e-6b09d1bdfaeb
15042024-11-18T18:40:31.043ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_81488621-5898-41d4-a112-d3e574fb2d33") failed: Property not found. retry in 1.036212486s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_81488621-5898-41d4-a112-d3e574fb2d33
15052024-11-18T18:40:31.092ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34") failed: Property not found. retry in 570.151983ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34
15062024-11-18T18:40:31.115ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6") failed: Property not found. retry in 908.71332ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6
15072024-11-18T18:40:31.213ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_efcbb893-1918-43b9-8315-e9bb90d664e8") failed: Property not found. retry in 843.325004ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_efcbb893-1918-43b9-8315-e9bb90d664e8
15082024-11-18T18:40:31.238ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60") failed: Property not found. retry in 1.338577574s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60
15092024-11-18T18:40:31.308ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1") failed: Property not found. retry in 787.423984ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1
15102024-11-18T18:40:31.343ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_023a09b8-2c2b-483c-be8c-540d1a35ce5a") failed: Property not found. retry in 787.604253ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_023a09b8-2c2b-483c-be8c-540d1a35ce5a
15112024-11-18T18:40:31.354ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_55ab4bd1-3e97-453c-b90c-176360dea573") failed: Property not found. retry in 1.3097285s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_55ab4bd1-3e97-453c-b90c-176360dea573
15122024-11-18T18:40:31.395ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d") failed: Property not found. retry in 1.377156211s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d
15132024-11-18T18:40:31.542ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1fa25dd7-d8f9-49ef-bb20-c6bad413aace") failed: Property not found. retry in 795.088549ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1fa25dd7-d8f9-49ef-bb20-c6bad413aace
15142024-11-18T18:40:31.637ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_ceea2287-3571-41ea-8f15-b2aeb4a74fd9") failed: Property not found. retry in 1.243908497s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_ceea2287-3571-41ea-8f15-b2aeb4a74fd9
15152024-11-18T18:40:31.675ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34") failed: Property not found. retry in 772.509246ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34
15162024-11-18T18:40:31.696ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d7ddf21e-0e1e-4004-b302-ecb81ba84c09") failed: Property not found. retry in 1.381556105s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d7ddf21e-0e1e-4004-b302-ecb81ba84c09
15172024-11-18T18:40:31.710ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46") failed: Property not found. retry in 1.008980746s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46
15182024-11-18T18:40:31.784ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e782f5fc-076e-414c-b87e-6b09d1bdfaeb") failed: Property not found. retry in 981.656821ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e782f5fc-076e-414c-b87e-6b09d1bdfaeb
15192024-11-18T18:40:31.865ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_6767b855-63e1-4a7d-ac38-c4e7eb56747f") failed: Property not found. retry in 1.141432186s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_6767b855-63e1-4a7d-ac38-c4e7eb56747f
15202024-11-18T18:40:31.887ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aad33dc-2f67-4a47-b973-90de986a5f65") failed: Property not found. retry in 1.40633526s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aad33dc-2f67-4a47-b973-90de986a5f65
15212024-11-18T18:40:31.927ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_10904f20-d090-41dc-9a54-dab632f01d4a") failed: Property not found. retry in 565.445619ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_10904f20-d090-41dc-9a54-dab632f01d4a
15222024-11-18T18:40:31.928ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:52003
15232024-11-18T18:40:31.930ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 697 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:52003 req_id = 50dd7740-d4a6-4632-8639-afc3f0f7ab3c response_code = 200 uri = /baseboard
15242024-11-18T18:40:32.018ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6062ae2a-331c-4cb6-a9b5-eb4310b33530") failed: Property not found. retry in 901.232298ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6062ae2a-331c-4cb6-a9b5-eb4310b33530
15252024-11-18T18:40:32.040ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6") failed: Property not found. retry in 798.330598ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6
15262024-11-18T18:40:32.072ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_efcbb893-1918-43b9-8315-e9bb90d664e8") failed: Property not found. retry in 1.248793338s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_efcbb893-1918-43b9-8315-e9bb90d664e8
15272024-11-18T18:40:32.095ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_81488621-5898-41d4-a112-d3e574fb2d33") failed: Property not found. retry in 1.148938144s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_81488621-5898-41d4-a112-d3e574fb2d33
15282024-11-18T18:40:32.110ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1") failed: Property not found. retry in 1.069033882s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1
15292024-11-18T18:40:32.145ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_023a09b8-2c2b-483c-be8c-540d1a35ce5a") failed: Property not found. retry in 1.472652178s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_023a09b8-2c2b-483c-be8c-540d1a35ce5a
15302024-11-18T18:40:32.192ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2735d4d6-073c-4860-ae50-e3742cc2cefa") failed: Property not found. retry in 1.236489909s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2735d4d6-073c-4860-ae50-e3742cc2cefa
15312024-11-18T18:40:32.355ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1fa25dd7-d8f9-49ef-bb20-c6bad413aace") failed: Property not found. retry in 1.029874751s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1fa25dd7-d8f9-49ef-bb20-c6bad413aace
15322024-11-18T18:40:32.465ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34") failed: Property not found. retry in 1.1656103s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34
15332024-11-18T18:40:32.510ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_10904f20-d090-41dc-9a54-dab632f01d4a") failed: Property not found. retry in 1.041342293s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_10904f20-d090-41dc-9a54-dab632f01d4a
15342024-11-18T18:40:32.594ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60") failed: Property not found. retry in 506.436213ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60
15352024-11-18T18:40:32.676ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_55ab4bd1-3e97-453c-b90c-176360dea573") failed: Property not found. retry in 1.105701338s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_55ab4bd1-3e97-453c-b90c-176360dea573
15362024-11-18T18:40:32.733ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46") failed: Property not found. retry in 814.664851ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46
15372024-11-18T18:40:32.782ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e782f5fc-076e-414c-b87e-6b09d1bdfaeb") failed: Property not found. retry in 756.644761ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e782f5fc-076e-414c-b87e-6b09d1bdfaeb
15382024-11-18T18:40:32.797ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d") failed: Property not found. retry in 560.259944ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d
15392024-11-18T18:40:32.849ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6") failed: Property not found. retry in 513.053659ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6
15402024-11-18T18:40:32.894ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_ceea2287-3571-41ea-8f15-b2aeb4a74fd9") failed: Property not found. retry in 1.453855416s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_ceea2287-3571-41ea-8f15-b2aeb4a74fd9
15412024-11-18T18:40:32.935ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6062ae2a-331c-4cb6-a9b5-eb4310b33530") failed: Property not found. retry in 1.172049891s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6062ae2a-331c-4cb6-a9b5-eb4310b33530
15422024-11-18T18:40:33.023ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_6767b855-63e1-4a7d-ac38-c4e7eb56747f") failed: Property not found. retry in 574.840097ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_6767b855-63e1-4a7d-ac38-c4e7eb56747f
15432024-11-18T18:40:33.095ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d7ddf21e-0e1e-4004-b302-ecb81ba84c09") failed: Property not found. retry in 1.354292846s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d7ddf21e-0e1e-4004-b302-ecb81ba84c09
15442024-11-18T18:40:33.116ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60") failed: Property not found. retry in 936.128285ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60
15452024-11-18T18:40:33.194ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1") failed: Property not found. retry in 1.406691917s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1
15462024-11-18T18:40:33.257ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_81488621-5898-41d4-a112-d3e574fb2d33") failed: Property not found. retry in 1.448268318s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_81488621-5898-41d4-a112-d3e574fb2d33
15472024-11-18T18:40:33.309ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aad33dc-2f67-4a47-b973-90de986a5f65") failed: Property not found. retry in 749.083418ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aad33dc-2f67-4a47-b973-90de986a5f65
15482024-11-18T18:40:33.340ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_efcbb893-1918-43b9-8315-e9bb90d664e8") failed: Property not found. retry in 672.342214ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_efcbb893-1918-43b9-8315-e9bb90d664e8
15492024-11-18T18:40:33.370ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d") failed: Property not found. retry in 1.286839394s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d
15502024-11-18T18:40:33.383ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6") failed: Property not found. retry in 1.399613622s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6
15512024-11-18T18:40:33.399ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1fa25dd7-d8f9-49ef-bb20-c6bad413aace") failed: Property not found. retry in 1.057475865s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1fa25dd7-d8f9-49ef-bb20-c6bad413aace
15522024-11-18T18:40:33.443ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2735d4d6-073c-4860-ae50-e3742cc2cefa") failed: Property not found. retry in 1.305965102s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2735d4d6-073c-4860-ae50-e3742cc2cefa
15532024-11-18T18:40:33.550ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e782f5fc-076e-414c-b87e-6b09d1bdfaeb") failed: Property not found. retry in 1.479031837s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e782f5fc-076e-414c-b87e-6b09d1bdfaeb
15542024-11-18T18:40:33.559ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46") failed: Property not found. retry in 1.031305767s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46
15552024-11-18T18:40:33.566ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_10904f20-d090-41dc-9a54-dab632f01d4a") failed: Property not found. retry in 1.406601038s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_10904f20-d090-41dc-9a54-dab632f01d4a
15562024-11-18T18:40:33.609ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_6767b855-63e1-4a7d-ac38-c4e7eb56747f") failed: Property not found. retry in 660.514744ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_6767b855-63e1-4a7d-ac38-c4e7eb56747f
15572024-11-18T18:40:33.632ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_023a09b8-2c2b-483c-be8c-540d1a35ce5a") failed: Property not found. retry in 945.291339ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_023a09b8-2c2b-483c-be8c-540d1a35ce5a
15582024-11-18T18:40:33.642ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34") failed: Property not found. retry in 541.375404ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34
15592024-11-18T18:40:33.795ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_55ab4bd1-3e97-453c-b90c-176360dea573") failed: Property not found. retry in 567.481639ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_55ab4bd1-3e97-453c-b90c-176360dea573
15602024-11-18T18:40:34.028ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_efcbb893-1918-43b9-8315-e9bb90d664e8") failed: Property not found. retry in 993.056389ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_efcbb893-1918-43b9-8315-e9bb90d664e8
15612024-11-18T18:40:34.067ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60") failed: Property not found. retry in 785.906563ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60
15622024-11-18T18:40:34.079ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aad33dc-2f67-4a47-b973-90de986a5f65") failed: Property not found. retry in 1.105033868s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aad33dc-2f67-4a47-b973-90de986a5f65
15632024-11-18T18:40:34.119ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6062ae2a-331c-4cb6-a9b5-eb4310b33530") failed: Property not found. retry in 792.921732ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6062ae2a-331c-4cb6-a9b5-eb4310b33530
15642024-11-18T18:40:34.194ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34") failed: Property not found. retry in 793.664479ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34
15652024-11-18T18:40:34.284ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_6767b855-63e1-4a7d-ac38-c4e7eb56747f") failed: Property not found. retry in 620.01934ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_6767b855-63e1-4a7d-ac38-c4e7eb56747f
15662024-11-18T18:40:34.361ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_ceea2287-3571-41ea-8f15-b2aeb4a74fd9") failed: Property not found. retry in 966.713273ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_ceea2287-3571-41ea-8f15-b2aeb4a74fd9
15672024-11-18T18:40:34.376ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_55ab4bd1-3e97-453c-b90c-176360dea573") failed: Property not found. retry in 937.939061ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_55ab4bd1-3e97-453c-b90c-176360dea573
15682024-11-18T18:40:34.463ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d7ddf21e-0e1e-4004-b302-ecb81ba84c09") failed: Property not found. retry in 1.463125294s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d7ddf21e-0e1e-4004-b302-ecb81ba84c09
15692024-11-18T18:40:34.472ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1fa25dd7-d8f9-49ef-bb20-c6bad413aace") failed: Property not found. retry in 1.340795528s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1fa25dd7-d8f9-49ef-bb20-c6bad413aace
15702024-11-18T18:40:34.591ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_023a09b8-2c2b-483c-be8c-540d1a35ce5a") failed: Property not found. retry in 1.02411559s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_023a09b8-2c2b-483c-be8c-540d1a35ce5a
15712024-11-18T18:40:34.602ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46") failed: Property not found. retry in 801.947712ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46
15722024-11-18T18:40:34.614ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1") failed: Property not found. retry in 980.478383ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1
15732024-11-18T18:40:34.671ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d") failed: Property not found. retry in 853.909742ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d
15742024-11-18T18:40:34.719ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_81488621-5898-41d4-a112-d3e574fb2d33") failed: Property not found. retry in 763.402087ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_81488621-5898-41d4-a112-d3e574fb2d33
15752024-11-18T18:40:34.765ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2735d4d6-073c-4860-ae50-e3742cc2cefa") failed: Property not found. retry in 738.261141ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2735d4d6-073c-4860-ae50-e3742cc2cefa
15762024-11-18T18:40:34.795ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6") failed: Property not found. retry in 828.080503ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6
15772024-11-18T18:40:34.869ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60") failed: Property not found. retry in 1.020186231s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60
15782024-11-18T18:40:34.920ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_6767b855-63e1-4a7d-ac38-c4e7eb56747f") failed: Property not found. retry in 792.968521ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_6767b855-63e1-4a7d-ac38-c4e7eb56747f
15792024-11-18T18:40:34.931ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6062ae2a-331c-4cb6-a9b5-eb4310b33530") failed: Property not found. retry in 608.404817ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6062ae2a-331c-4cb6-a9b5-eb4310b33530
15802024-11-18T18:40:34.985ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_10904f20-d090-41dc-9a54-dab632f01d4a") failed: Property not found. retry in 1.365392444s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_10904f20-d090-41dc-9a54-dab632f01d4a
15812024-11-18T18:40:35.001ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34") failed: Property not found. retry in 1.097397693s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34
15822024-11-18T18:40:35.034ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_efcbb893-1918-43b9-8315-e9bb90d664e8") failed: Property not found. retry in 1.123837393s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_efcbb893-1918-43b9-8315-e9bb90d664e8
15832024-11-18T18:40:35.050ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e782f5fc-076e-414c-b87e-6b09d1bdfaeb") failed: Property not found. retry in 1.229486612s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e782f5fc-076e-414c-b87e-6b09d1bdfaeb
15842024-11-18T18:40:35.199ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aad33dc-2f67-4a47-b973-90de986a5f65") failed: Property not found. retry in 647.287656ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aad33dc-2f67-4a47-b973-90de986a5f65
15852024-11-18T18:40:35.328ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_55ab4bd1-3e97-453c-b90c-176360dea573") failed: Property not found. retry in 610.257927ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_55ab4bd1-3e97-453c-b90c-176360dea573
15862024-11-18T18:40:35.347ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_ceea2287-3571-41ea-8f15-b2aeb4a74fd9") failed: Property not found. retry in 1.281175589s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_ceea2287-3571-41ea-8f15-b2aeb4a74fd9
15872024-11-18T18:40:35.418ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46") failed: Property not found. retry in 1.142601657s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46
15882024-11-18T18:40:35.501ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_81488621-5898-41d4-a112-d3e574fb2d33") failed: Property not found. retry in 1.463545462s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_81488621-5898-41d4-a112-d3e574fb2d33
15892024-11-18T18:40:35.518ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2735d4d6-073c-4860-ae50-e3742cc2cefa") failed: Property not found. retry in 746.769679ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2735d4d6-073c-4860-ae50-e3742cc2cefa
15902024-11-18T18:40:35.538ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d") failed: Property not found. retry in 1.186737369s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d
15912024-11-18T18:40:35.555ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6062ae2a-331c-4cb6-a9b5-eb4310b33530") failed: Property not found. retry in 604.502599ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6062ae2a-331c-4cb6-a9b5-eb4310b33530
15922024-11-18T18:40:35.610ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1") failed: Property not found. retry in 632.664348ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1
15932024-11-18T18:40:35.631ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_023a09b8-2c2b-483c-be8c-540d1a35ce5a") failed: Property not found. retry in 561.444656ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_023a09b8-2c2b-483c-be8c-540d1a35ce5a
15942024-11-18T18:40:35.643ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6") failed: Property not found. retry in 775.698203ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6
15952024-11-18T18:40:35.727ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_6767b855-63e1-4a7d-ac38-c4e7eb56747f") failed: Property not found. retry in 1.19462138s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_6767b855-63e1-4a7d-ac38-c4e7eb56747f
15962024-11-18T18:40:35.827ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1fa25dd7-d8f9-49ef-bb20-c6bad413aace") failed: Property not found. retry in 823.861064ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1fa25dd7-d8f9-49ef-bb20-c6bad413aace
15972024-11-18T18:40:35.861ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aad33dc-2f67-4a47-b973-90de986a5f65") failed: Property not found. retry in 1.409856319s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aad33dc-2f67-4a47-b973-90de986a5f65
15982024-11-18T18:40:35.901ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60") failed: Property not found. retry in 1.156141455s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60
15992024-11-18T18:40:35.941ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d7ddf21e-0e1e-4004-b302-ecb81ba84c09") failed: Property not found. retry in 953.948077ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d7ddf21e-0e1e-4004-b302-ecb81ba84c09
16002024-11-18T18:40:35.955ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_55ab4bd1-3e97-453c-b90c-176360dea573") failed: Property not found. retry in 1.021509255s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_55ab4bd1-3e97-453c-b90c-176360dea573
16012024-11-18T18:40:36.115ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34") failed: Property not found. retry in 1.308434501s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34
16022024-11-18T18:40:36.171ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_efcbb893-1918-43b9-8315-e9bb90d664e8") failed: Property not found. retry in 1.433624195s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_efcbb893-1918-43b9-8315-e9bb90d664e8
16032024-11-18T18:40:36.187ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6062ae2a-331c-4cb6-a9b5-eb4310b33530") failed: Property not found. retry in 1.240264921s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6062ae2a-331c-4cb6-a9b5-eb4310b33530
16042024-11-18T18:40:36.207ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_023a09b8-2c2b-483c-be8c-540d1a35ce5a") failed: Property not found. retry in 1.255291107s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_023a09b8-2c2b-483c-be8c-540d1a35ce5a
16052024-11-18T18:40:36.259ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1") failed: Property not found. retry in 597.621197ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1
16062024-11-18T18:40:36.279ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2735d4d6-073c-4860-ae50-e3742cc2cefa") failed: Property not found. retry in 701.255949ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2735d4d6-073c-4860-ae50-e3742cc2cefa
16072024-11-18T18:40:36.295ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e782f5fc-076e-414c-b87e-6b09d1bdfaeb") failed: Property not found. retry in 1.281034987s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e782f5fc-076e-414c-b87e-6b09d1bdfaeb
16082024-11-18T18:40:36.369ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_10904f20-d090-41dc-9a54-dab632f01d4a") failed: Property not found. retry in 589.965143ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_10904f20-d090-41dc-9a54-dab632f01d4a
16092024-11-18T18:40:36.436ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6") failed: Property not found. retry in 930.251566ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6
16102024-11-18T18:40:36.576ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46") failed: Property not found. retry in 761.629466ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46
16112024-11-18T18:40:36.640ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_ceea2287-3571-41ea-8f15-b2aeb4a74fd9") failed: Property not found. retry in 574.541869ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_ceea2287-3571-41ea-8f15-b2aeb4a74fd9
16122024-11-18T18:40:36.661ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1fa25dd7-d8f9-49ef-bb20-c6bad413aace") failed: Property not found. retry in 710.073326ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1fa25dd7-d8f9-49ef-bb20-c6bad413aace
16132024-11-18T18:40:36.737ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d") failed: Property not found. retry in 1.251062928s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d
16142024-11-18T18:40:36.868ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1") failed: Property not found. retry in 681.985722ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1
16152024-11-18T18:40:36.905ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d7ddf21e-0e1e-4004-b302-ecb81ba84c09") failed: Property not found. retry in 647.0236ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d7ddf21e-0e1e-4004-b302-ecb81ba84c09
16162024-11-18T18:40:36.934ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_6767b855-63e1-4a7d-ac38-c4e7eb56747f") failed: Property not found. retry in 1.204259592s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_6767b855-63e1-4a7d-ac38-c4e7eb56747f
16172024-11-18T18:40:36.969ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_10904f20-d090-41dc-9a54-dab632f01d4a") failed: Property not found. retry in 1.355352719s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_10904f20-d090-41dc-9a54-dab632f01d4a
16182024-11-18T18:40:36.978ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_81488621-5898-41d4-a112-d3e574fb2d33") failed: Property not found. retry in 1.072744339s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_81488621-5898-41d4-a112-d3e574fb2d33
16192024-11-18T18:40:36.987ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_55ab4bd1-3e97-453c-b90c-176360dea573") failed: Property not found. retry in 973.012989ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_55ab4bd1-3e97-453c-b90c-176360dea573
16202024-11-18T18:40:36.996ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2735d4d6-073c-4860-ae50-e3742cc2cefa") failed: Property not found. retry in 1.101113903s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2735d4d6-073c-4860-ae50-e3742cc2cefa
16212024-11-18T18:40:37.067ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60") failed: Property not found. retry in 1.262681249s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60
16222024-11-18T18:40:37.225ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_ceea2287-3571-41ea-8f15-b2aeb4a74fd9") failed: Property not found. retry in 565.530707ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_ceea2287-3571-41ea-8f15-b2aeb4a74fd9
16232024-11-18T18:40:37.281ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3aad33dc-2f67-4a47-b973-90de986a5f65") failed: Property not found. retry in 1.39095479s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3aad33dc-2f67-4a47-b973-90de986a5f65
16242024-11-18T18:40:37.348ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46") failed: Property not found. retry in 1.444386331s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46
16252024-11-18T18:40:37.383ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6") failed: Property not found. retry in 929.077636ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6
16262024-11-18T18:40:37.433ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34") failed: Property not found. retry in 834.566567ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34
16272024-11-18T18:40:37.441ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6062ae2a-331c-4cb6-a9b5-eb4310b33530") failed: Property not found. retry in 1.215393905s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6062ae2a-331c-4cb6-a9b5-eb4310b33530
16282024-11-18T18:40:37.471ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_023a09b8-2c2b-483c-be8c-540d1a35ce5a") failed: Property not found. retry in 1.378641573s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_023a09b8-2c2b-483c-be8c-540d1a35ce5a
16292024-11-18T18:40:37.560ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1") failed: Property not found. retry in 685.709653ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1
16302024-11-18T18:40:37.568ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d7ddf21e-0e1e-4004-b302-ecb81ba84c09") failed: Property not found. retry in 1.036904618s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d7ddf21e-0e1e-4004-b302-ecb81ba84c09
16312024-11-18T18:40:37.586ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_e782f5fc-076e-414c-b87e-6b09d1bdfaeb") failed: Property not found. retry in 1.24335507s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_e782f5fc-076e-414c-b87e-6b09d1bdfaeb
16322024-11-18T18:40:37.614ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_efcbb893-1918-43b9-8315-e9bb90d664e8") failed: Property not found. retry in 560.020796ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_efcbb893-1918-43b9-8315-e9bb90d664e8
16332024-11-18T18:40:37.801ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_ceea2287-3571-41ea-8f15-b2aeb4a74fd9") failed: Property not found. retry in 1.357801054s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_ceea2287-3571-41ea-8f15-b2aeb4a74fd9
16342024-11-18T18:40:37.970ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_55ab4bd1-3e97-453c-b90c-176360dea573") failed: Property not found. retry in 624.660916ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_55ab4bd1-3e97-453c-b90c-176360dea573
16352024-11-18T18:40:38.001ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d") failed: Property not found. retry in 550.071548ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d
16362024-11-18T18:40:38.063ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_81488621-5898-41d4-a112-d3e574fb2d33") failed: Property not found. retry in 544.703693ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_81488621-5898-41d4-a112-d3e574fb2d33
16372024-11-18T18:40:38.108ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_2735d4d6-073c-4860-ae50-e3742cc2cefa") failed: Property not found. retry in 1.380711026s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_2735d4d6-073c-4860-ae50-e3742cc2cefa
16382024-11-18T18:40:38.148ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_6767b855-63e1-4a7d-ac38-c4e7eb56747f") failed: Property not found. retry in 997.692896ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_6767b855-63e1-4a7d-ac38-c4e7eb56747f
16392024-11-18T18:40:38.184ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_efcbb893-1918-43b9-8315-e9bb90d664e8") failed: Property not found. retry in 772.260103ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_efcbb893-1918-43b9-8315-e9bb90d664e8
16402024-11-18T18:40:38.256ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1") failed: Property not found. retry in 1.328803229s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1
16412024-11-18T18:40:38.282ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34") failed: Property not found. retry in 598.218927ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34
16422024-11-18T18:40:38.325ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6") failed: Property not found. retry in 507.740078ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6
16432024-11-18T18:40:38.336ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_10904f20-d090-41dc-9a54-dab632f01d4a") failed: Property not found. retry in 726.232633ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_10904f20-d090-41dc-9a54-dab632f01d4a
16442024-11-18T18:40:38.346ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60") failed: Property not found. retry in 1.441771682s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60
16452024-11-18T18:40:38.605ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_55ab4bd1-3e97-453c-b90c-176360dea573") failed: Property not found. retry in 1.217528167s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_55ab4bd1-3e97-453c-b90c-176360dea573
16462024-11-18T18:40:39.877ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3311 zone = oxz_crucible_efcbb893-1918-43b9-8315-e9bb90d664e8
16472024-11-18T18:40:39.877ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3311 zone = oxz_nexus_1bd1422d-700f-41e4-b299-7268d8edd3b1
16482024-11-18T18:40:39.877ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3311 zone = oxz_crucible_81488621-5898-41d4-a112-d3e574fb2d33
16492024-11-18T18:40:39.877ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3311 zone = oxz_clickhouse_55ab4bd1-3e97-453c-b90c-176360dea573
16502024-11-18T18:40:39.877ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3311 zone = oxz_nexus_88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d
16512024-11-18T18:40:39.877ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3311 zone = oxz_external_dns_ceea2287-3571-41ea-8f15-b2aeb4a74fd9
16522024-11-18T18:40:39.877ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3311 zone = oxz_nexus_d7ddf21e-0e1e-4004-b302-ecb81ba84c09
16532024-11-18T18:40:39.877ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3311 zone = oxz_oximeter_8716f96b-ee60-45b3-a886-a68fcf5530d6
16542024-11-18T18:40:39.877ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3311 zone = oxz_crucible_e782f5fc-076e-414c-b87e-6b09d1bdfaeb
16552024-11-18T18:40:39.877ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3311 zone = oxz_crucible_10904f20-d090-41dc-9a54-dab632f01d4a
16562024-11-18T18:40:39.877ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3311 zone = oxz_crucible_pantry_1fa25dd7-d8f9-49ef-bb20-c6bad413aace
16572024-11-18T18:40:39.877ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3311 zone = oxz_crucible_bb775b2d-b688-494a-8509-79ecf4eb8f60
16582024-11-18T18:40:39.877ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3311 zone = oxz_crucible_3aad33dc-2f67-4a47-b973-90de986a5f65
16592024-11-18T18:40:39.877ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3311 zone = oxz_crucible_2735d4d6-073c-4860-ae50-e3742cc2cefa
16602024-11-18T18:40:39.877ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3311 zone = oxz_crucible_6062ae2a-331c-4cb6-a9b5-eb4310b33530
16612024-11-18T18:40:39.877ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3311 zone = oxz_external_dns_6767b855-63e1-4a7d-ac38-c4e7eb56747f
16622024-11-18T18:40:39.877ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3311 zone = oxz_crucible_pantry_023a09b8-2c2b-483c-be8c-540d1a35ce5a
16632024-11-18T18:40:39.877ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3311 zone = oxz_crucible_pantry_b2e5ebc5-bab5-4bcb-9e47-6ecea60a3e46
16642024-11-18T18:40:39.877ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3311 zone = oxz_crucible_396921f6-50a5-459a-86ad-182888d80a34
16652024-11-18T18:40:39.878ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/9216f0f9-29c9-4d33-8c71-0f237e1331a0/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
16662024-11-18T18:40:39.891ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/c23ca4a6-dcd8-4835-a278-26664c492a4d/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
16672024-11-18T18:40:39.903ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 38010105 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:60003 req_id = 0a3ad6a7-f03a-4cff-a0cb-1b6a2b759afb response_code = 204 uri = /omicron-zones
16682024-11-18T18:40:39.925ZINFOSledAgent (RSS): Finished setting up services file = sled-agent/src/rack_setup/service.rs:1448
16692024-11-18T18:40:39.925ZINFOSledAgent (RSS): Writing ledger to /pool/int/9216f0f9-29c9-4d33-8c71-0f237e1331a0/config/.rss-plan-completed.marker.tmp file = common/src/ledger.rs:199
16702024-11-18T18:40:39.925ZINFOSledAgent (RSS): Writing ledger to /pool/int/c23ca4a6-dcd8-4835-a278-26664c492a4d/config/.rss-plan-completed.marker.tmp file = common/src/ledger.rs:199
16712024-11-18T18:40:40.160ZINFOSledAgent (RSS): Handing off control to Nexus file = sled-agent/src/rack_setup/service.rs:787
16722024-11-18T18:40:40.199ZINFOSledAgent (RSS): Nexus address: [fd00:1122:3344:101::a]:12221 file = sled-agent/src/rack_setup/service.rs:799
16732024-11-18T18:40:40.251ZINFOSledAgent (RSS): rack_network_config: RackNetworkConfigV2 { bfd: [], bgp: [], infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [ PortConfigV2 { addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.2.174, width: 24, }, ), vlan_id: None, }, ], autoneg: false, bgp_peers: [], lldp: None, port: "qsfp0", routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), nexthop: 10.151.2.100, rib_priority: None, vlan_id: None, }, ], switch: Switch0, tx_eq: None, uplink_port_fec: Some( None, ), uplink_port_speed: Speed40G, }, ], rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, } file = sled-agent/src/rack_setup/service.rs:1013
16742024-11-18T18:40:42.743ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/a5c736b2-440f-4c70-9b42-8f18e4718901/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1075
16752024-11-18T18:40:42.804ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/a5c736b2-440f-4c70-9b42-8f18e4718901/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1075
16762024-11-18T18:40:42.947ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/a5c736b2-440f-4c70-9b42-8f18e4718901/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1075
16772024-11-18T18:40:43.066ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/a5c736b2-440f-4c70-9b42-8f18e4718901/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1075
16782024-11-18T18:40:43.307ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/a5c736b2-440f-4c70-9b42-8f18e4718901/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1075
16792024-11-18T18:40:43.507ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/a5c736b2-440f-4c70-9b42-8f18e4718901/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1075
16802024-11-18T18:40:43.662ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/a5c736b2-440f-4c70-9b42-8f18e4718901/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1075
16812024-11-18T18:40:43.820ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/a5c736b2-440f-4c70-9b42-8f18e4718901/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1075
16822024-11-18T18:40:44.056ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/a5c736b2-440f-4c70-9b42-8f18e4718901/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1075
16832024-11-18T18:40:44.336ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/a5c736b2-440f-4c70-9b42-8f18e4718901/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1075
16842024-11-18T18:40:44.618ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/a5c736b2-440f-4c70-9b42-8f18e4718901/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1075
16852024-11-18T18:40:45.004ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/a5c736b2-440f-4c70-9b42-8f18e4718901/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1075
16862024-11-18T18:40:45.907ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/a5c736b2-440f-4c70-9b42-8f18e4718901/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1075
16872024-11-18T18:40:46.698ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/a5c736b2-440f-4c70-9b42-8f18e4718901/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1075
16882024-11-18T18:40:47.671ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "9c80eb7f-eeff-4169-9a0d-2c956cfcaeda", "content-length": "173", "date": "Mon, 18 Nov 2024 18:40:47 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: rack with id \"a5c736b2-440f-4c70-9b42-8f18e4718901\"", request_id: "9c80eb7f-eeff-4169-9a0d-2c956cfcaeda" } file = sled-agent/src/rack_setup/service.rs:1075
16892024-11-18T18:40:48.435ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "946e574c-0519-4b59-b4fe-de64bfbb927c", "content-length": "173", "date": "Mon, 18 Nov 2024 18:40:48 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: rack with id \"a5c736b2-440f-4c70-9b42-8f18e4718901\"", request_id: "946e574c-0519-4b59-b4fe-de64bfbb927c" } file = sled-agent/src/rack_setup/service.rs:1075
16902024-11-18T18:40:51.348ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "ba2413fd-91da-43b1-a7de-8b77eb58a270", "content-length": "173", "date": "Mon, 18 Nov 2024 18:40:51 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"ee2b3f44-b73f-4358-9df7-edb19f5079e2\"", request_id: "ba2413fd-91da-43b1-a7de-8b77eb58a270" } file = sled-agent/src/rack_setup/service.rs:1075
16912024-11-18T18:40:54.315ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "63b9b30c-028d-4073-9c60-d9bdba554f44", "content-length": "173", "date": "Mon, 18 Nov 2024 18:40:53 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"ee2b3f44-b73f-4358-9df7-edb19f5079e2\"", request_id: "63b9b30c-028d-4073-9c60-d9bdba554f44" } file = sled-agent/src/rack_setup/service.rs:1075
16922024-11-18T18:40:57.761ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "2f4ad22e-cecb-40df-b54c-26a8b4018f48", "content-length": "173", "date": "Mon, 18 Nov 2024 18:40:57 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"ee2b3f44-b73f-4358-9df7-edb19f5079e2\"", request_id: "2f4ad22e-cecb-40df-b54c-26a8b4018f48" } file = sled-agent/src/rack_setup/service.rs:1075
16932024-11-18T18:41:01.572ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "19ef2688-5001-4e83-ba46-f862fbc61612", "content-length": "173", "date": "Mon, 18 Nov 2024 18:41:01 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"ee2b3f44-b73f-4358-9df7-edb19f5079e2\"", request_id: "19ef2688-5001-4e83-ba46-f862fbc61612" } file = sled-agent/src/rack_setup/service.rs:1075
16942024-11-18T18:41:01.967ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:55491
16952024-11-18T18:41:01.967ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 43 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:55491 req_id = 3b22b59a-b0ef-4555-91fc-0e3f7e10f277 response_code = 200 uri = /baseboard
16962024-11-18T18:41:05.307ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "c2e58ec9-95d0-4eb6-89b5-9f7d289a6c2b", "content-length": "173", "date": "Mon, 18 Nov 2024 18:41:04 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"ee2b3f44-b73f-4358-9df7-edb19f5079e2\"", request_id: "c2e58ec9-95d0-4eb6-89b5-9f7d289a6c2b" } file = sled-agent/src/rack_setup/service.rs:1075
16972024-11-18T18:41:08.605ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "9bcd144a-d6f9-40e3-9d76-36b31c058dbc", "content-length": "173", "date": "Mon, 18 Nov 2024 18:41:08 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"ee2b3f44-b73f-4358-9df7-edb19f5079e2\"", request_id: "9bcd144a-d6f9-40e3-9d76-36b31c058dbc" } file = sled-agent/src/rack_setup/service.rs:1075
16982024-11-18T18:41:09.000ZINFOSledAgent (NexusNotifierTask): Successfully put SledAgentInfo to nexus file = sled-agent/src/nexus.rs:500 new_generation = 1 old_generation = 1 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
16992024-11-18T18:41:09.103ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:47797
17002024-11-18T18:41:09.104ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
17012024-11-18T18:41:09.104ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
17022024-11-18T18:41:09.104ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/9216f0f9-29c9-4d33-8c71-0f237e1331a0/config/omicron-zones.json file = common/src/ledger.rs:177
17032024-11-18T18:41:09.107ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/c23ca4a6-dcd8-4835-a278-26664c492a4d/config/omicron-zones.json file = common/src/ledger.rs:177
17042024-11-18T18:41:09.120ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
17052024-11-18T18:41:09.120ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_4be98a0c-ab4c-4309-948b-04d776603876
17062024-11-18T18:41:09.146ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
17072024-11-18T18:41:09.146ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_4f35fd7e-936a-4e09-946b-7cb174580897
17082024-11-18T18:41:09.165ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
17092024-11-18T18:41:09.165ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_df4faf1b-f747-4cda-8481-deaa8523a533
17102024-11-18T18:41:09.188ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 0199d4fd-4090-4a7c-83e6-040814eb79bb (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
17112024-11-18T18:41:09.188ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_0199d4fd-4090-4a7c-83e6-040814eb79bb
17122024-11-18T18:41:09.209ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 5840d858-3782-45e1-a6e0-d60d070ad2c3 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
17132024-11-18T18:41:09.209ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_5840d858-3782-45e1-a6e0-d60d070ad2c3
17142024-11-18T18:41:09.231ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 131b4d69-dbb0-4397-9284-176e4a3c973d (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
17152024-11-18T18:41:09.231ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_131b4d69-dbb0-4397-9284-176e4a3c973d
17162024-11-18T18:41:09.251ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 453f7245-5f70-4fe8-b37e-054e56d5c1c5 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
17172024-11-18T18:41:09.251ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_453f7245-5f70-4fe8-b37e-054e56d5c1c5
17182024-11-18T18:41:09.273ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: b09b6e80-5341-4ded-a520-3f312403e198 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
17192024-11-18T18:41:09.273ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_b09b6e80-5341-4ded-a520-3f312403e198
17202024-11-18T18:41:09.296ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: f23dc95a-d65a-4537-b65e-161954e059fa (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
17212024-11-18T18:41:09.296ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_f23dc95a-d65a-4537-b65e-161954e059fa
17222024-11-18T18:41:09.308ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 204631 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:47797 req_id = 51844c49-6fb6-407e-bf6b-10f92ed66fde response_code = 200 uri = /inventory
17232024-11-18T18:41:16.238ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:38246
17242024-11-18T18:41:16.240ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:731 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 6a2febaf-2bbc-4cc0-aab6-80daf994ec33, kind: Service { id: 6767b855-63e1-4a7d-ac38-c4e7eb56747f }, name: Name("external-dns-6767b855-63e1-4a7d-ac38-c4e7eb56747f"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 152, 29])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 72590ac9-2812-4929-9b37-1829b4168efa, kind: Service { id: ceea2287-3571-41ea-8f15-b2aeb4a74fd9 }, name: Name("external-dns-ceea2287-3571-41ea-8f15-b2aeb4a74fd9"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 170, 19])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: ab6c5d8c-f7d0-4cf6-8c3c-5acf8f3959a9, kind: Service { id: d7ddf21e-0e1e-4004-b302-ecb81ba84c09 }, name: Name("nexus-d7ddf21e-0e1e-4004-b302-ecb81ba84c09"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 151, 123])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d0f6d114-ab1d-41fb-a2bb-c154f579e21f, kind: Service { id: 88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d }, name: Name("nexus-88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 37])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f6afceac-9bde-4010-ba9d-6c2eebc86b2d, kind: Service { id: 1bd1422d-700f-41e4-b299-7268d8edd3b1 }, name: Name("nexus-1bd1422d-700f-41e4-b299-7268d8edd3b1"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 0])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
17252024-11-18T18:41:16.240ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
17262024-11-18T18:41:16.241ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
17272024-11-18T18:41:16.241ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17282024-11-18T18:41:16.241ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte0" rules = []
17292024-11-18T18:41:16.241ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17302024-11-18T18:41:16.241ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17312024-11-18T18:41:16.241ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2391 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:38246 req_id = 0e4086f3-30ba-4d82-9f60-093ff32e3d2d response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
17322024-11-18T18:41:16.242ZINFOSledAgent (RSS): Handoff to Nexus is complete file = sled-agent/src/rack_setup/service.rs:1085
17332024-11-18T18:41:16.385ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:54144
17342024-11-18T18:41:16.386ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:731 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 6a2febaf-2bbc-4cc0-aab6-80daf994ec33, kind: Service { id: 6767b855-63e1-4a7d-ac38-c4e7eb56747f }, name: Name("external-dns-6767b855-63e1-4a7d-ac38-c4e7eb56747f"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 152, 29])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 72590ac9-2812-4929-9b37-1829b4168efa, kind: Service { id: ceea2287-3571-41ea-8f15-b2aeb4a74fd9 }, name: Name("external-dns-ceea2287-3571-41ea-8f15-b2aeb4a74fd9"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 170, 19])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: ab6c5d8c-f7d0-4cf6-8c3c-5acf8f3959a9, kind: Service { id: d7ddf21e-0e1e-4004-b302-ecb81ba84c09 }, name: Name("nexus-d7ddf21e-0e1e-4004-b302-ecb81ba84c09"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 151, 123])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d0f6d114-ab1d-41fb-a2bb-c154f579e21f, kind: Service { id: 88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d }, name: Name("nexus-88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 37])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f6afceac-9bde-4010-ba9d-6c2eebc86b2d, kind: Service { id: 1bd1422d-700f-41e4-b299-7268d8edd3b1 }, name: Name("nexus-1bd1422d-700f-41e4-b299-7268d8edd3b1"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 0])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
17352024-11-18T18:41:16.386ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
17362024-11-18T18:41:16.386ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
17372024-11-18T18:41:16.386ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17382024-11-18T18:41:16.386ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte0" rules = []
17392024-11-18T18:41:16.386ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17402024-11-18T18:41:16.386ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17412024-11-18T18:41:16.386ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 315 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:54144 req_id = b00c177d-69fd-421c-8ffb-711fdb4bbf4b response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
17422024-11-18T18:41:16.452ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:37267
17432024-11-18T18:41:16.452ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
17442024-11-18T18:41:16.452ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
17452024-11-18T18:41:16.452ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/9216f0f9-29c9-4d33-8c71-0f237e1331a0/config/omicron-zones.json file = common/src/ledger.rs:177
17462024-11-18T18:41:16.453ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/c23ca4a6-dcd8-4835-a278-26664c492a4d/config/omicron-zones.json file = common/src/ledger.rs:177
17472024-11-18T18:41:16.465ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
17482024-11-18T18:41:16.465ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_4be98a0c-ab4c-4309-948b-04d776603876
17492024-11-18T18:41:16.492ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
17502024-11-18T18:41:16.492ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_4f35fd7e-936a-4e09-946b-7cb174580897
17512024-11-18T18:41:16.517ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
17522024-11-18T18:41:16.517ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_df4faf1b-f747-4cda-8481-deaa8523a533
17532024-11-18T18:41:16.543ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 0199d4fd-4090-4a7c-83e6-040814eb79bb (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
17542024-11-18T18:41:16.543ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_0199d4fd-4090-4a7c-83e6-040814eb79bb
17552024-11-18T18:41:16.569ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 5840d858-3782-45e1-a6e0-d60d070ad2c3 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
17562024-11-18T18:41:16.569ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_5840d858-3782-45e1-a6e0-d60d070ad2c3
17572024-11-18T18:41:16.591ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 131b4d69-dbb0-4397-9284-176e4a3c973d (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
17582024-11-18T18:41:16.591ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_131b4d69-dbb0-4397-9284-176e4a3c973d
17592024-11-18T18:41:16.614ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 453f7245-5f70-4fe8-b37e-054e56d5c1c5 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
17602024-11-18T18:41:16.614ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_453f7245-5f70-4fe8-b37e-054e56d5c1c5
17612024-11-18T18:41:16.640ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: b09b6e80-5341-4ded-a520-3f312403e198 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
17622024-11-18T18:41:16.640ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_b09b6e80-5341-4ded-a520-3f312403e198
17632024-11-18T18:41:16.664ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: f23dc95a-d65a-4537-b65e-161954e059fa (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
17642024-11-18T18:41:16.664ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_f23dc95a-d65a-4537-b65e-161954e059fa
17652024-11-18T18:41:16.676ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 224405 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:37267 req_id = 0edab739-fe5c-40a4-9581-ae3075601252 response_code = 200 uri = /inventory
17662024-11-18T18:41:16.982ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:42918
17672024-11-18T18:41:16.982ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:731 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 6a2febaf-2bbc-4cc0-aab6-80daf994ec33, kind: Service { id: 6767b855-63e1-4a7d-ac38-c4e7eb56747f }, name: Name("external-dns-6767b855-63e1-4a7d-ac38-c4e7eb56747f"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 152, 29])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 72590ac9-2812-4929-9b37-1829b4168efa, kind: Service { id: ceea2287-3571-41ea-8f15-b2aeb4a74fd9 }, name: Name("external-dns-ceea2287-3571-41ea-8f15-b2aeb4a74fd9"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 170, 19])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: ab6c5d8c-f7d0-4cf6-8c3c-5acf8f3959a9, kind: Service { id: d7ddf21e-0e1e-4004-b302-ecb81ba84c09 }, name: Name("nexus-d7ddf21e-0e1e-4004-b302-ecb81ba84c09"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 151, 123])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d0f6d114-ab1d-41fb-a2bb-c154f579e21f, kind: Service { id: 88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d }, name: Name("nexus-88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 37])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f6afceac-9bde-4010-ba9d-6c2eebc86b2d, kind: Service { id: 1bd1422d-700f-41e4-b299-7268d8edd3b1 }, name: Name("nexus-1bd1422d-700f-41e4-b299-7268d8edd3b1"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 0])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
17682024-11-18T18:41:16.982ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
17692024-11-18T18:41:16.982ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
17702024-11-18T18:41:16.982ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17712024-11-18T18:41:16.982ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte0" rules = []
17722024-11-18T18:41:16.983ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17732024-11-18T18:41:16.983ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17742024-11-18T18:41:16.983ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 324 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:42918 req_id = 63c20dfc-8759-4878-b680-57e42834e3ea response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
17752024-11-18T18:41:17.545ZINFOSledAgent (dropshot): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:48709 remote_addr = [fd00:1122:3344:101::d]:38778 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
17762024-11-18T18:41:17.569ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 23113 local_addr = [fd00:1122:3344:101::1]:48709 method = GET remote_addr = [fd00:1122:3344:101::d]:38778 req_id = 51c14703-d74d-45a3-80f3-ca91e930cd2c response_code = 200 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 uri = /ee2b3f44-b73f-4358-9df7-edb19f5079e2
17772024-11-18T18:41:17.580ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:59682
17782024-11-18T18:41:17.580ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:731 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 6a2febaf-2bbc-4cc0-aab6-80daf994ec33, kind: Service { id: 6767b855-63e1-4a7d-ac38-c4e7eb56747f }, name: Name("external-dns-6767b855-63e1-4a7d-ac38-c4e7eb56747f"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 152, 29])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 72590ac9-2812-4929-9b37-1829b4168efa, kind: Service { id: ceea2287-3571-41ea-8f15-b2aeb4a74fd9 }, name: Name("external-dns-ceea2287-3571-41ea-8f15-b2aeb4a74fd9"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 170, 19])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: ab6c5d8c-f7d0-4cf6-8c3c-5acf8f3959a9, kind: Service { id: d7ddf21e-0e1e-4004-b302-ecb81ba84c09 }, name: Name("nexus-d7ddf21e-0e1e-4004-b302-ecb81ba84c09"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 151, 123])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d0f6d114-ab1d-41fb-a2bb-c154f579e21f, kind: Service { id: 88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d }, name: Name("nexus-88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 37])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f6afceac-9bde-4010-ba9d-6c2eebc86b2d, kind: Service { id: 1bd1422d-700f-41e4-b299-7268d8edd3b1 }, name: Name("nexus-1bd1422d-700f-41e4-b299-7268d8edd3b1"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 0])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
17792024-11-18T18:41:17.580ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
17802024-11-18T18:41:17.580ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
17812024-11-18T18:41:17.580ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17822024-11-18T18:41:17.580ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte0" rules = []
17832024-11-18T18:41:17.580ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17842024-11-18T18:41:17.580ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17852024-11-18T18:41:17.580ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 274 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:59682 req_id = 3229786a-df6a-4638-b723-437f6437f168 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
17862024-11-18T18:41:18.469ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:61439
17872024-11-18T18:41:18.469ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 697 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:61439 req_id = a8e5247c-c53a-4073-a4ee-aa06153ad63b response_code = 200 uri = /vpc-routes
17882024-11-18T18:41:18.573ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
17892024-11-18T18:41:18.573ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 228 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:61439 req_id = 4d982dd2-4d7c-427b-9461-447b5de1dd46 response_code = 204 uri = /eip-gateways
17902024-11-18T18:41:18.650ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
17912024-11-18T18:41:18.743ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:44796
17922024-11-18T18:41:18.743ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
17932024-11-18T18:41:18.829ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
17942024-11-18T18:41:18.900ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 156602 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:44796 req_id = 32feb20f-497c-4359-9f9d-6b4475693ee3 response_code = 204 uri = /switch-ports
17952024-11-18T18:41:18.901ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 842 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:44796 req_id = 6f3a553f-61b6-442d-805b-78b0a0c7e83a response_code = 200 uri = /network-bootstore-config
17962024-11-18T18:41:18.910ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:56232
17972024-11-18T18:41:18.911ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 375 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:56232 req_id = d7a2fef6-959a-46d7-81b6-d0965a2620a1 response_code = 200 uri = /v2p
17982024-11-18T18:41:18.953ZINFOSledAgent (bootstore): Attempting to update network config with generation: 2, current_generation: 1 file = bootstore/src/schemes/v0/peer.rs:564 peer_id = pc-sock-i86pc
17992024-11-18T18:41:18.953ZINFOSledAgent (bootstore): Writing ledger to /pool/int/9216f0f9-29c9-4d33-8c71-0f237e1331a0/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
18002024-11-18T18:41:18.954ZINFOSledAgent (bootstore): Writing ledger to /pool/int/c23ca4a6-dcd8-4835-a278-26664c492a4d/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
18012024-11-18T18:41:18.956ZINFOSledAgent (bootstore): Broadcasting network config with generation 2 file = bootstore/src/schemes/v0/peer.rs:648 peer_id = pc-sock-i86pc
18022024-11-18T18:41:18.956ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3306 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:44796 req_id = e0dd9810-d4a4-4afe-8cef-365f47ee4066 response_code = 204 uri = /network-bootstore-config
18032024-11-18T18:41:18.982ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3301 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:61439 req_id = 9cc5f399-eaf3-45b3-95db-aba4d8bd260a response_code = 204 uri = /vpc-routes
18042024-11-18T18:41:19.751ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:32984
18052024-11-18T18:41:19.751ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 96 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:32984 req_id = dc8e55a0-c386-4f0c-a33d-80b3b11b98be response_code = 200 uri = /vpc-routes
18062024-11-18T18:41:19.890ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:38872
18072024-11-18T18:41:19.890ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
18082024-11-18T18:41:19.931ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
18092024-11-18T18:41:19.992ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 101753 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:38872 req_id = 815dd030-231b-4632-8179-8192a623b853 response_code = 204 uri = /switch-ports
18102024-11-18T18:41:19.992ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:52686
18112024-11-18T18:41:19.992ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 99 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:52686 req_id = 7d83d8c4-5a52-49a0-afbe-66dec37b74c4 response_code = 200 uri = /vpc-routes
18122024-11-18T18:41:19.993ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 213 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:38872 req_id = db301f20-b436-4066-b5d0-d766fc4b0770 response_code = 200 uri = /network-bootstore-config
18132024-11-18T18:41:19.994ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:37740
18142024-11-18T18:41:19.994ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 173 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:37740 req_id = b230300c-4b6b-47b3-b659-68b953f60996 response_code = 200 uri = /v2p
18152024-11-18T18:41:20.066ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
18162024-11-18T18:41:20.066ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 91 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:52686 req_id = 2e04da40-69ba-41d1-9907-896cbd227425 response_code = 204 uri = /eip-gateways
18172024-11-18T18:41:20.078ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
18182024-11-18T18:41:20.078ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 57 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:32984 req_id = c549c56d-f72e-463d-ad07-d2cbf8e17b46 response_code = 204 uri = /eip-gateways
18192024-11-18T18:41:20.079ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 172 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:52686 req_id = 02f7f10a-bc71-46d7-9f0a-7044d7dd1d99 response_code = 204 uri = /vpc-routes
18202024-11-18T18:41:20.081ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:52083
18212024-11-18T18:41:20.081ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
18222024-11-18T18:41:20.129ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
18232024-11-18T18:41:20.189ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 108016 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:52083 req_id = 9de6e644-a957-4fd0-b014-d39283e3835c response_code = 204 uri = /switch-ports
18242024-11-18T18:41:20.190ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 272 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:52083 req_id = 2d522ee4-6ba7-4bfe-b861-7b025d0740e8 response_code = 200 uri = /network-bootstore-config
18252024-11-18T18:41:20.190ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 432 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:32984 req_id = 9b9acf95-9813-487c-ab1c-2e0c052c29ea response_code = 204 uri = /vpc-routes
18262024-11-18T18:41:20.197ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:48759
18272024-11-18T18:41:20.197ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 101 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:48759 req_id = 2cf44a11-e783-4f2c-80b7-e95aaf731834 response_code = 200 uri = /v2p
18282024-11-18T18:41:25.374ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:48486
18292024-11-18T18:41:25.374ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 94 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:48486 req_id = fa442ca0-378e-4a44-994b-94c1c775439f response_code = 200 uri = /vpc-routes
18302024-11-18T18:41:25.434ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
18312024-11-18T18:41:25.434ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 57 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:48486 req_id = a67aa8f0-e27c-432c-893f-79b270e03ac8 response_code = 204 uri = /eip-gateways
18322024-11-18T18:41:25.442ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 161 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:48486 req_id = c4e47f73-5add-4dfe-823e-b77a7501fa24 response_code = 204 uri = /vpc-routes
18332024-11-18T18:41:32.003ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:42216
18342024-11-18T18:41:32.003ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 64 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:42216 req_id = 1f229f9b-b87d-467c-be9f-60789d228b54 response_code = 200 uri = /baseboard
18352024-11-18T18:41:48.733ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:64093
18362024-11-18T18:41:48.733ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 46 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:64093 req_id = f934ac64-c7ff-4808-9162-4782f10793e8 response_code = 200 uri = /vpc-routes
18372024-11-18T18:41:48.774ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:36953
18382024-11-18T18:41:48.774ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 127 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:36953 req_id = b09471ba-bef4-47b1-896c-366f8c7a2fd4 response_code = 200 uri = /v2p
18392024-11-18T18:41:48.808ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:57978
18402024-11-18T18:41:48.808ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
18412024-11-18T18:41:48.845ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
18422024-11-18T18:41:48.904ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 96544 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:57978 req_id = 3f1ff7f3-5c86-4b1d-a0c7-cef0a19000fc response_code = 204 uri = /switch-ports
18432024-11-18T18:41:48.904ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 103 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:57978 req_id = b4fa0547-ea30-42fa-bf3f-7e660e70be71 response_code = 200 uri = /network-bootstore-config
18442024-11-18T18:41:48.993ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
18452024-11-18T18:41:48.993ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 68 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:64093 req_id = a194e26f-059e-4f2f-8ea6-85a958074fda response_code = 204 uri = /eip-gateways
18462024-11-18T18:41:49.104ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 233 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:64093 req_id = cae5a1f8-0bb4-4b0b-9064-a0c095e6e3c1 response_code = 204 uri = /vpc-routes
18472024-11-18T18:41:49.709ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:44882
18482024-11-18T18:41:49.709ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 56 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:44882 req_id = 2c5ec61d-8f6a-41c1-8b5f-ecd491c6d6af response_code = 200 uri = /vpc-routes
18492024-11-18T18:41:49.827ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:63527
18502024-11-18T18:41:49.827ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 46 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:63527 req_id = 2228687c-4211-4216-a01f-6c83f7732887 response_code = 200 uri = /vpc-routes
18512024-11-18T18:41:49.878ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:57246
18522024-11-18T18:41:49.878ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 120 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:57246 req_id = 4aef6fe7-e6f4-451e-bf6d-c6de29be1c9c response_code = 200 uri = /v2p
18532024-11-18T18:41:49.919ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
18542024-11-18T18:41:49.919ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 63 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:63527 req_id = 51edb235-e374-4d91-ba0f-e140d118b5a2 response_code = 204 uri = /eip-gateways
18552024-11-18T18:41:49.926ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:57816
18562024-11-18T18:41:49.926ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 113 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:57816 req_id = fe5d8ac1-85df-46a1-9035-1d2e05047982 response_code = 200 uri = /v2p
18572024-11-18T18:41:49.974ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
18582024-11-18T18:41:49.974ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 44 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:44882 req_id = 3e183f92-df18-4cc6-9986-cfa7d5ebc592 response_code = 204 uri = /eip-gateways
18592024-11-18T18:41:49.977ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:59615
18602024-11-18T18:41:49.977ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
18612024-11-18T18:41:50.015ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
18622024-11-18T18:41:50.072ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 95275 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:59615 req_id = 3e66f301-e2d1-46dd-bed3-e585553d6310 response_code = 204 uri = /switch-ports
18632024-11-18T18:41:50.072ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:57768
18642024-11-18T18:41:50.073ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
18652024-11-18T18:41:50.073ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 284 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:59615 req_id = 4a2062d2-679e-4a8e-8dd1-1ce19b9b80f4 response_code = 200 uri = /network-bootstore-config
18662024-11-18T18:41:50.073ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 462 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:63527 req_id = acf0480c-7da2-423e-a0b8-b9a6747af284 response_code = 204 uri = /vpc-routes
18672024-11-18T18:41:50.078ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 5389 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:44882 req_id = 9014d7d0-1183-4043-b0c3-2c7e3ea73ca5 response_code = 204 uri = /vpc-routes
18682024-11-18T18:41:50.113ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
18692024-11-18T18:41:50.169ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 96726 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:57768 req_id = e59fd3b4-3bae-4821-ad35-ece9aa6b96df response_code = 204 uri = /switch-ports
18702024-11-18T18:41:50.170ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 152 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:57768 req_id = fa2f7e58-7adc-4e69-8cb3-73382a59d918 response_code = 200 uri = /network-bootstore-config
18712024-11-18T18:42:02.038ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:33408
18722024-11-18T18:42:02.038ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 69 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:33408 req_id = 6d813bd0-b154-4b3d-8695-33397ba72ea8 response_code = 200 uri = /baseboard
18732024-11-18T18:42:18.548ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:62903
18742024-11-18T18:42:18.548ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 89 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:62903 req_id = 6411b728-60f8-4a15-9712-81e5b9503723 response_code = 200 uri = /vpc-routes
18752024-11-18T18:42:18.644ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
18762024-11-18T18:42:18.644ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 76 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:62903 req_id = a80bef91-8f1d-48d8-801d-0e650f96d5af response_code = 204 uri = /eip-gateways
18772024-11-18T18:42:18.665ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:52504
18782024-11-18T18:42:18.665ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
18792024-11-18T18:42:18.706ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
18802024-11-18T18:42:18.765ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 99850 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:52504 req_id = fea31528-f96c-4f2c-9de3-8462000d3c2e response_code = 204 uri = /switch-ports
18812024-11-18T18:42:18.765ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:63434
18822024-11-18T18:42:18.765ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 264 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:52504 req_id = d33e7cdf-11d5-437d-a906-ac10caf50f29 response_code = 200 uri = /network-bootstore-config
18832024-11-18T18:42:18.765ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 227 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:63434 req_id = 6c90a471-4c52-4f9b-865d-2882cea68592 response_code = 200 uri = /v2p
18842024-11-18T18:42:18.847ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 321 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:62903 req_id = 521a9ae8-1a2d-40d1-a6b8-5b0ecee4ca1d response_code = 204 uri = /vpc-routes
18852024-11-18T18:42:19.524ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:53736
18862024-11-18T18:42:19.524ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 47 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:53736 req_id = e14480cc-1fb7-4c45-a1cc-e6886ee86d73 response_code = 200 uri = /vpc-routes
18872024-11-18T18:42:19.602ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
18882024-11-18T18:42:19.602ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 70 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:53736 req_id = cbb54077-26b3-4dae-9690-b6f305cba5f1 response_code = 204 uri = /eip-gateways
18892024-11-18T18:42:19.614ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:36671
18902024-11-18T18:42:19.614ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
18912024-11-18T18:42:19.653ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
18922024-11-18T18:42:19.714ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 99619 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:36671 req_id = 6eef7bc1-1273-4400-a18e-6fec523b321d response_code = 204 uri = /switch-ports
18932024-11-18T18:42:19.714ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51894
18942024-11-18T18:42:19.714ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:58213
18952024-11-18T18:42:19.714ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:40693
18962024-11-18T18:42:19.715ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 357 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:36671 req_id = 0d348717-ab30-45a3-be4c-0813f09bea32 response_code = 200 uri = /network-bootstore-config
18972024-11-18T18:42:19.715ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 157 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:40693 req_id = c684b994-53ca-427d-85ad-fafa0016f932 response_code = 200 uri = /vpc-routes
18982024-11-18T18:42:19.715ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 306 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:51894 req_id = 23c63210-79d7-4928-ad36-d7aad9250a26 response_code = 200 uri = /v2p
18992024-11-18T18:42:19.715ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 211 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:58213 req_id = a699ebb7-f16c-49f1-8072-4ad6b2527b79 response_code = 200 uri = /v2p
19002024-11-18T18:42:19.729ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 218 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:53736 req_id = 55d44688-abbb-4cce-9586-1eac04713496 response_code = 204 uri = /vpc-routes
19012024-11-18T18:42:19.762ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:59289
19022024-11-18T18:42:19.762ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
19032024-11-18T18:42:19.802ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
19042024-11-18T18:42:19.857ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 94142 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:59289 req_id = dc819fd9-de09-43ad-8af9-8037630df6b2 response_code = 204 uri = /switch-ports
19052024-11-18T18:42:19.857ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 147 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:59289 req_id = 8f51d41e-3dc9-42ab-b74a-e2dabb603a1a response_code = 200 uri = /network-bootstore-config
19062024-11-18T18:42:19.857ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
19072024-11-18T18:42:19.857ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 121 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:40693 req_id = 18dba76b-e2f9-4554-9bff-49ca706367d8 response_code = 204 uri = /eip-gateways
19082024-11-18T18:42:19.869ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 260 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:40693 req_id = 3befa305-1b2a-47c2-9203-017255ddb46e response_code = 204 uri = /vpc-routes
19092024-11-18T18:42:32.070ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:48678
19102024-11-18T18:42:32.071ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 73 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:48678 req_id = 6328b8f6-bfbe-4311-8114-8a23235b8d2f response_code = 200 uri = /baseboard
19112024-11-18T18:42:48.623ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:39062
19122024-11-18T18:42:48.623ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 67 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:39062 req_id = 348a8d1f-f2ea-431d-a856-1bb525725f34 response_code = 200 uri = /vpc-routes
19132024-11-18T18:42:48.760ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:55860
19142024-11-18T18:42:48.760ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
19152024-11-18T18:42:48.798ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
19162024-11-18T18:42:48.849ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 88396 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:55860 req_id = 38d3828a-d57f-4a35-8537-bb4a854fd3db response_code = 204 uri = /switch-ports
19172024-11-18T18:42:48.849ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:33352
19182024-11-18T18:42:48.849ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
19192024-11-18T18:42:48.849ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 222 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:39062 req_id = a24b6330-b1aa-42ac-80a7-4e5bcdb548e6 response_code = 204 uri = /eip-gateways
19202024-11-18T18:42:48.849ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 298 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:55860 req_id = de9a9f0c-128c-4f88-aa1c-5329f4104d44 response_code = 200 uri = /network-bootstore-config
19212024-11-18T18:42:48.849ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 226 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:33352 req_id = 30d2ab2f-abc6-448f-8de5-683071892a52 response_code = 200 uri = /v2p
19222024-11-18T18:42:48.965ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 182 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:39062 req_id = 7774f20e-18fc-444f-b037-70e8aa9c5308 response_code = 204 uri = /vpc-routes
19232024-11-18T18:42:49.691ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:34696
19242024-11-18T18:42:49.691ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 106 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:34696 req_id = 64c41bc3-2238-47a7-99cc-650272e5f8ac response_code = 200 uri = /vpc-routes
19252024-11-18T18:42:49.734ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:55924
19262024-11-18T18:42:49.734ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 42 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:55924 req_id = 9bb4d9a2-0b4d-4918-959c-49afee68137d response_code = 200 uri = /vpc-routes
19272024-11-18T18:42:49.833ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
19282024-11-18T18:42:49.833ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 86 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:55924 req_id = 785929ed-c375-4a13-9346-307bdc17d77b response_code = 204 uri = /eip-gateways
19292024-11-18T18:42:49.844ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 204 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:55924 req_id = da8c67d3-6b0e-4e46-9d29-7a5bfbba9f4a response_code = 204 uri = /vpc-routes
19302024-11-18T18:42:49.889ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
19312024-11-18T18:42:49.889ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 53 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:34696 req_id = a4dc2c8d-b92c-4079-a8e2-0fa9605598cb response_code = 204 uri = /eip-gateways
19322024-11-18T18:42:49.893ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51506
19332024-11-18T18:42:49.893ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 128 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:51506 req_id = 25ac9d67-8784-4b15-95c8-235f081c25f3 response_code = 200 uri = /v2p
19342024-11-18T18:42:49.900ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 163 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:34696 req_id = 10e630b5-7614-4193-af95-84a5371b7688 response_code = 204 uri = /vpc-routes
19352024-11-18T18:42:50.009ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:46642
19362024-11-18T18:42:50.009ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 143 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:46642 req_id = ec3bb3e0-2459-4f8f-a2e3-123ffe078c49 response_code = 200 uri = /v2p
19372024-11-18T18:42:50.114ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:40050
19382024-11-18T18:42:50.114ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
19392024-11-18T18:42:50.152ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
19402024-11-18T18:42:50.210ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 95961 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:40050 req_id = aebf1680-5763-41d3-b5c1-342e7004e6fa response_code = 204 uri = /switch-ports
19412024-11-18T18:42:50.210ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:35761
19422024-11-18T18:42:50.210ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
19432024-11-18T18:42:50.210ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 243 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:40050 req_id = acd758c1-2bca-4a3d-af09-deb022b5ad0b response_code = 200 uri = /network-bootstore-config
19442024-11-18T18:42:50.247ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
19452024-11-18T18:42:50.298ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 87856 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:35761 req_id = fca43a1f-2ff1-496a-9ebd-a1eb0d86dc32 response_code = 204 uri = /switch-ports
19462024-11-18T18:42:50.299ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 155 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:35761 req_id = 29c5d72d-f714-47ef-ad88-cadc4da4e189 response_code = 200 uri = /network-bootstore-config
19472024-11-18T18:43:02.130ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:39543
19482024-11-18T18:43:02.130ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 55 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:39543 req_id = e68b54a9-b3be-4e20-b30a-b5544dbf8c78 response_code = 200 uri = /baseboard
19492024-11-18T18:43:18.619ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:62293
19502024-11-18T18:43:18.619ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 78 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:62293 req_id = 8e17b080-c206-44ad-91a7-37db8f475fe2 response_code = 200 uri = /vpc-routes
19512024-11-18T18:43:18.652ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:34528
19522024-11-18T18:43:18.652ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
19532024-11-18T18:43:18.695ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
19542024-11-18T18:43:18.757ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 105321 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:34528 req_id = 6dd104bc-22e8-4e13-b06f-81847a836434 response_code = 204 uri = /switch-ports
19552024-11-18T18:43:18.761ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 135 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:34528 req_id = e032a9ee-4758-4250-8b5f-dffc7c90b97a response_code = 200 uri = /network-bootstore-config
19562024-11-18T18:43:18.781ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
19572024-11-18T18:43:18.781ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 88 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:62293 req_id = 38dfadb8-47e0-4aff-b15b-e1af7ef2676c response_code = 204 uri = /eip-gateways
19582024-11-18T18:43:18.886ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 188 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:62293 req_id = 69ad22d0-215e-41c5-a01b-b6d8670fd681 response_code = 204 uri = /vpc-routes
19592024-11-18T18:43:18.928ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:38780
19602024-11-18T18:43:18.928ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 90 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:38780 req_id = 45f3e7aa-1de6-4f84-b371-a007a5220f7a response_code = 200 uri = /v2p
19612024-11-18T18:43:19.508ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:45036
19622024-11-18T18:43:19.508ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 86 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:45036 req_id = a552f9ed-14b0-48cf-a870-e785cab204b1 response_code = 200 uri = /vpc-routes
19632024-11-18T18:43:19.568ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:45543
19642024-11-18T18:43:19.568ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 153 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:45543 req_id = e7e4a241-5da6-44e2-9e6c-1efa430215d9 response_code = 200 uri = /v2p
19652024-11-18T18:43:19.619ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:49460
19662024-11-18T18:43:19.619ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
19672024-11-18T18:43:19.664ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
19682024-11-18T18:43:19.727ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 107692 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:49460 req_id = 3f3cf256-1211-46c4-aff2-7f360bab4bdf response_code = 204 uri = /switch-ports
19692024-11-18T18:43:19.727ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:39134
19702024-11-18T18:43:19.727ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:44177
19712024-11-18T18:43:19.727ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
19722024-11-18T18:43:19.727ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 227 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:45036 req_id = 005662c0-0679-4b5a-a047-57f9366df7f8 response_code = 204 uri = /eip-gateways
19732024-11-18T18:43:19.727ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 287 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:49460 req_id = 7d196c84-c97e-43e6-a62e-fb7e487d9283 response_code = 200 uri = /network-bootstore-config
19742024-11-18T18:43:19.727ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
19752024-11-18T18:43:19.733ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 5351 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:39134 req_id = 23115426-1775-4df7-97bf-b419467b8155 response_code = 200 uri = /vpc-routes
19762024-11-18T18:43:19.738ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 348 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:45036 req_id = 7fb95509-0cd1-4d52-8b20-4985f7d41666 response_code = 204 uri = /vpc-routes
19772024-11-18T18:43:19.761ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
19782024-11-18T18:43:19.761ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 778 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:39134 req_id = 3bf7f336-2af7-4b46-8e26-eaba4251995d response_code = 204 uri = /eip-gateways
19792024-11-18T18:43:19.771ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
19802024-11-18T18:43:19.771ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 280 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:39134 req_id = 51951093-1f4d-4081-9502-a90bbe766c69 response_code = 204 uri = /vpc-routes
19812024-11-18T18:43:19.833ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 105804 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:44177 req_id = ee75ee49-1328-446e-b398-f9126e6a2ae8 response_code = 204 uri = /switch-ports
19822024-11-18T18:43:19.833ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 146 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:44177 req_id = a0e6ebdf-1de5-4b1d-a76f-ff9ca00804c8 response_code = 200 uri = /network-bootstore-config
19832024-11-18T18:43:19.915ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:38192
19842024-11-18T18:43:19.915ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 96 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:38192 req_id = ba24e781-aec3-4108-9916-bdd978ebba6e response_code = 200 uri = /v2p
19852024-11-18T18:43:32.162ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:63716
19862024-11-18T18:43:32.162ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 55 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:63716 req_id = 02a32f69-01fa-4dd9-9e88-1e364f0c789d response_code = 200 uri = /baseboard
19872024-11-18T18:43:48.715ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:34870
19882024-11-18T18:43:48.715ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 57 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:34870 req_id = 2b2e27b8-3a1e-4611-b5c1-a7fb53adfa68 response_code = 200 uri = /vpc-routes
19892024-11-18T18:43:48.795ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
19902024-11-18T18:43:48.795ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 78 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:34870 req_id = 2b8d41da-2625-4043-b13c-cb9a45276902 response_code = 204 uri = /eip-gateways
19912024-11-18T18:43:48.901ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 167 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:34870 req_id = f0a2ac96-e2e7-4c1e-859f-e051f5e790e5 response_code = 204 uri = /vpc-routes
19922024-11-18T18:43:48.909ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:52281
19932024-11-18T18:43:48.910ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 96 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:52281 req_id = 77f9556e-b79a-47c7-8aa8-729c33a5b701 response_code = 200 uri = /v2p
19942024-11-18T18:43:48.926ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:62061
19952024-11-18T18:43:48.926ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
19962024-11-18T18:43:48.964ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
19972024-11-18T18:43:49.020ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 94027 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:62061 req_id = 28028817-036e-4021-9291-e94f3ba3c5fe response_code = 204 uri = /switch-ports
19982024-11-18T18:43:49.020ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 224 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:62061 req_id = 96ca6e03-64ac-4bd0-832d-eaa984587330 response_code = 200 uri = /network-bootstore-config
19992024-11-18T18:43:49.495ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:39808
20002024-11-18T18:43:49.495ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 58 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:39808 req_id = 2969bdc9-eba7-4326-8dc6-ddc0b9994455 response_code = 200 uri = /vpc-routes
20012024-11-18T18:43:49.618ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
20022024-11-18T18:43:49.618ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 62 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:39808 req_id = db376473-ad9e-495a-a68f-0541ff7abdd2 response_code = 204 uri = /eip-gateways
20032024-11-18T18:43:49.640ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 277 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:39808 req_id = dc5bf08a-3da9-46aa-9654-b3c073a97ccc response_code = 204 uri = /vpc-routes
20042024-11-18T18:43:49.813ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:41463
20052024-11-18T18:43:49.813ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 121 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:41463 req_id = c07f19ee-443c-4db7-931d-986a52f2ae44 response_code = 200 uri = /v2p
20062024-11-18T18:43:49.914ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:38357
20072024-11-18T18:43:49.914ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 38 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:38357 req_id = 2567e1d6-e7eb-4294-b902-d1b6b04ddd0a response_code = 200 uri = /vpc-routes
20082024-11-18T18:43:49.952ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:36289
20092024-11-18T18:43:49.952ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
20102024-11-18T18:43:49.991ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
20112024-11-18T18:43:50.051ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 99145 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:36289 req_id = 79f29d8f-dd1c-4073-8e6d-5ed677dd6602 response_code = 204 uri = /switch-ports
20122024-11-18T18:43:50.052ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:54410
20132024-11-18T18:43:50.052ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 252 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:36289 req_id = e8ca5047-f37b-4601-bca0-c796f296ffa5 response_code = 200 uri = /network-bootstore-config
20142024-11-18T18:43:50.052ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 202 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:54410 req_id = 6fc51655-508a-4d2a-a92b-895cd1929d7f response_code = 200 uri = /v2p
20152024-11-18T18:43:50.073ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:55099
20162024-11-18T18:43:50.073ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
20172024-11-18T18:43:50.110ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
20182024-11-18T18:43:50.168ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 94754 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:55099 req_id = 4fbabf92-995d-4be2-9fa5-8348e7be9fce response_code = 204 uri = /switch-ports
20192024-11-18T18:43:50.168ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
20202024-11-18T18:43:50.168ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 212 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:38357 req_id = c87a3594-79e4-4823-b6b4-7036f735a35e response_code = 204 uri = /eip-gateways
20212024-11-18T18:43:50.168ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 280 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:55099 req_id = 6a131048-61ca-4f55-8060-5e803ceff2ea response_code = 200 uri = /network-bootstore-config
20222024-11-18T18:43:50.179ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 220 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:38357 req_id = 84fdfada-5583-4063-8ed6-d8a01457dad2 response_code = 204 uri = /vpc-routes
20232024-11-18T18:44:02.194ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:44294
20242024-11-18T18:44:02.194ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 70 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:44294 req_id = 060b62af-1734-406d-b0a0-95d9154f319f response_code = 200 uri = /baseboard
20252024-11-18T18:44:18.599ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:51159
20262024-11-18T18:44:18.600ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 83 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:51159 req_id = 94061ccd-8a56-47b3-9332-5cb4a541a2cc response_code = 200 uri = /vpc-routes
20272024-11-18T18:44:18.693ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
20282024-11-18T18:44:18.693ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 89 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:51159 req_id = 4e8c529a-9caf-4c4e-abd4-dd290eec5e89 response_code = 204 uri = /eip-gateways
20292024-11-18T18:44:18.702ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 167 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:51159 req_id = 01ddf4cf-d2ef-431d-bdca-2b30b0cab554 response_code = 204 uri = /vpc-routes
20302024-11-18T18:44:18.746ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:54664
20312024-11-18T18:44:18.746ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
20322024-11-18T18:44:18.787ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
20332024-11-18T18:44:18.842ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 96169 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:54664 req_id = 75368616-32a3-40f6-922d-718d32c5e049 response_code = 204 uri = /switch-ports
20342024-11-18T18:44:18.842ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 211 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:54664 req_id = 70576ec5-c23a-46d6-92fd-aed9742bab4d response_code = 200 uri = /network-bootstore-config
20352024-11-18T18:44:18.852ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:48929
20362024-11-18T18:44:18.852ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 121 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:48929 req_id = d91a3eea-1f8f-4215-a3ce-479759354031 response_code = 200 uri = /v2p
20372024-11-18T18:44:19.592ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:47192
20382024-11-18T18:44:19.593ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 135 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:47192 req_id = ac287602-1e1e-471d-90f3-d836b8338ab1 response_code = 200 uri = /v2p
20392024-11-18T18:44:19.610ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:34934
20402024-11-18T18:44:19.610ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 36 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:34934 req_id = f4c4b864-c633-4017-b637-bdcb448a8bf2 response_code = 200 uri = /vpc-routes
20412024-11-18T18:44:19.629ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:45964
20422024-11-18T18:44:19.629ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
20432024-11-18T18:44:19.669ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
20442024-11-18T18:44:19.727ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 97872 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:45964 req_id = 6d46d554-ccc1-49af-98fc-fa2e158a3736 response_code = 204 uri = /switch-ports
20452024-11-18T18:44:19.727ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:48382
20462024-11-18T18:44:19.727ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
20472024-11-18T18:44:19.727ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 125 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:34934 req_id = 50767dc2-47d7-4bbd-9a02-f3f49397b74d response_code = 204 uri = /eip-gateways
20482024-11-18T18:44:19.727ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 87 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:48382 req_id = c75ce0d0-632e-451a-b923-16a62a374ffa response_code = 200 uri = /vpc-routes
20492024-11-18T18:44:19.727ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 233 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:45964 req_id = 3f0bc5fb-833f-4c29-b51a-12990272e0c0 response_code = 200 uri = /network-bootstore-config
20502024-11-18T18:44:19.740ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 212 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:34934 req_id = d990f703-80d0-460d-816a-a3e9b2d463c3 response_code = 204 uri = /vpc-routes
20512024-11-18T18:44:19.751ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
20522024-11-18T18:44:19.751ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 33 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:48382 req_id = a4d238ac-4f2d-410d-a945-18191d3ecec8 response_code = 204 uri = /eip-gateways
20532024-11-18T18:44:19.760ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 207 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:48382 req_id = 5eea77d2-13f2-40bf-8cc3-7cd68a325d38 response_code = 204 uri = /vpc-routes
20542024-11-18T18:44:19.856ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:48712
20552024-11-18T18:44:19.856ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
20562024-11-18T18:44:19.898ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
20572024-11-18T18:44:19.958ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 101991 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:48712 req_id = c48953cf-a7d8-4588-b152-19b9f11a4bba response_code = 204 uri = /switch-ports
20582024-11-18T18:44:19.958ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:63086
20592024-11-18T18:44:19.959ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 263 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:48712 req_id = 22ff4c5e-e7bf-4c42-98a7-15bb25f38553 response_code = 200 uri = /network-bootstore-config
20602024-11-18T18:44:19.959ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 285 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:63086 req_id = 77ce1259-e095-4601-a03c-4dba5f7ad16a response_code = 200 uri = /v2p
20612024-11-18T18:44:32.228ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:58117
20622024-11-18T18:44:32.228ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 66 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:58117 req_id = 303e0acc-2732-47ca-8c5c-31056182c859 response_code = 200 uri = /baseboard
20632024-11-18T18:44:48.774ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:52872
20642024-11-18T18:44:48.775ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 62 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:52872 req_id = 4a0c476a-8fe5-4903-8d35-61ed35b3bf55 response_code = 200 uri = /vpc-routes
20652024-11-18T18:44:48.969ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
20662024-11-18T18:44:48.969ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 75 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:52872 req_id = 19040438-aa6c-4a02-8c28-52573db53921 response_code = 204 uri = /eip-gateways
20672024-11-18T18:44:48.979ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 168 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:52872 req_id = 2fcda96b-1bdb-45df-acbe-a0ae62376547 response_code = 204 uri = /vpc-routes
20682024-11-18T18:44:49.021ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:39461
20692024-11-18T18:44:49.021ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 102 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:39461 req_id = 5e4ea2f3-8be7-4a71-8ed5-1d9f905abf2f response_code = 200 uri = /v2p
20702024-11-18T18:44:49.023ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:51805
20712024-11-18T18:44:49.023ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
20722024-11-18T18:44:49.061ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
20732024-11-18T18:44:49.117ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 94559 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:51805 req_id = 1c608f75-9b10-4edc-9dbe-a6fab8af10d0 response_code = 204 uri = /switch-ports
20742024-11-18T18:44:49.124ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 79 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:51805 req_id = e9542a1e-e647-4b22-978b-92b77f60bd1e response_code = 200 uri = /network-bootstore-config
20752024-11-18T18:44:49.652ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:48196
20762024-11-18T18:44:49.652ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 49 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:48196 req_id = c9b08756-75a3-4316-a3ab-d589b90faf5b response_code = 200 uri = /vpc-routes
20772024-11-18T18:44:49.674ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
20782024-11-18T18:44:49.674ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 67 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:48196 req_id = dc326ac7-5488-41da-a4f8-09f0c010cf79 response_code = 204 uri = /eip-gateways
20792024-11-18T18:44:49.685ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 220 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:48196 req_id = 44402f6d-94a9-4056-9a92-34ea20f22451 response_code = 204 uri = /vpc-routes
20802024-11-18T18:44:49.742ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:53190
20812024-11-18T18:44:49.742ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 87 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:53190 req_id = 4e8eefa9-5f1a-4c35-b4a0-651f505716aa response_code = 200 uri = /vpc-routes
20822024-11-18T18:44:49.804ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:45345
20832024-11-18T18:44:49.805ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 136 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:45345 req_id = 24b2916a-91d1-40a5-9b8e-976d2ebe49da response_code = 200 uri = /v2p
20842024-11-18T18:44:49.829ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:61723
20852024-11-18T18:44:49.829ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
20862024-11-18T18:44:49.870ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
20872024-11-18T18:44:49.940ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 111366 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:61723 req_id = 6ee0bd02-9aa4-4921-91ea-59642ca5d8fb response_code = 204 uri = /switch-ports
20882024-11-18T18:44:49.940ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:39328
20892024-11-18T18:44:49.941ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
20902024-11-18T18:44:49.941ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 234 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:53190 req_id = 661918f3-097b-44d9-94e8-dd01f6bb28a3 response_code = 204 uri = /eip-gateways
20912024-11-18T18:44:49.941ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 316 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:61723 req_id = 12bc5920-7b13-4bef-be87-afde60a891c4 response_code = 200 uri = /network-bootstore-config
20922024-11-18T18:44:49.941ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
20932024-11-18T18:44:49.972ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:53736
20942024-11-18T18:44:49.972ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 206 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:53736 req_id = 9f754876-073b-4a59-92a3-c72cf87eb157 response_code = 200 uri = /v2p
20952024-11-18T18:44:49.980ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
20962024-11-18T18:44:49.985ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 4679 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:53190 req_id = c1a3d104-ee44-4390-b2f7-73bc36bf012d response_code = 204 uri = /vpc-routes
20972024-11-18T18:44:50.040ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 99206 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:39328 req_id = c5fa8e9e-faf3-4b8e-b186-ecd0c061d206 response_code = 204 uri = /switch-ports
20982024-11-18T18:44:50.040ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 125 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:39328 req_id = 12043375-621f-4f47-ad93-43c23e03d13e response_code = 200 uri = /network-bootstore-config
20992024-11-18T18:45:02.265ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:53333
21002024-11-18T18:45:02.265ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 51 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:53333 req_id = 655447e4-9e54-41ce-9b72-e7e5bffe8752 response_code = 200 uri = /baseboard
21012024-11-18T18:45:18.673ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:58689
21022024-11-18T18:45:18.673ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 69 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:58689 req_id = 29617a7d-4074-44d9-9b41-850212a9eae5 response_code = 200 uri = /vpc-routes
21032024-11-18T18:45:18.684ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:46193
21042024-11-18T18:45:18.684ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
21052024-11-18T18:45:18.724ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
21062024-11-18T18:45:18.789ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 105386 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:46193 req_id = 33258c30-dafd-42e5-9fd6-c1b898fd45a6 response_code = 204 uri = /switch-ports
21072024-11-18T18:45:18.789ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:32775
21082024-11-18T18:45:18.789ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
21092024-11-18T18:45:18.789ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 184 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:58689 req_id = 11508de3-1e89-4dbd-a17c-a221414e3e43 response_code = 204 uri = /eip-gateways
21102024-11-18T18:45:18.790ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 219 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:46193 req_id = cd42beed-e82e-44aa-b332-d27cb35ccf21 response_code = 200 uri = /network-bootstore-config
21112024-11-18T18:45:18.790ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 236 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:32775 req_id = fde3d1c3-c489-47dc-a354-749bfb730a9f response_code = 200 uri = /v2p
21122024-11-18T18:45:18.799ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 204 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:58689 req_id = 1004e5d7-6658-4291-ad23-aef8bc93ff4b response_code = 204 uri = /vpc-routes
21132024-11-18T18:45:19.504ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:65530
21142024-11-18T18:45:19.504ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 38 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:65530 req_id = 464f59ef-7a17-4c07-a37e-969710883fdd response_code = 200 uri = /vpc-routes
21152024-11-18T18:45:19.567ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
21162024-11-18T18:45:19.567ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 57 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:65530 req_id = 77c752fb-2982-4741-acce-6554ede0044a response_code = 204 uri = /eip-gateways
21172024-11-18T18:45:19.577ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 178 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:65530 req_id = 2fc19ea2-53f6-4b78-8125-5d5526a6770b response_code = 204 uri = /vpc-routes
21182024-11-18T18:45:19.592ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:60197
21192024-11-18T18:45:19.593ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 173 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:60197 req_id = 9e4d77e4-ab31-4376-a92e-8ebcadec398c response_code = 200 uri = /v2p
21202024-11-18T18:45:19.673ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:58451
21212024-11-18T18:45:19.674ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 37 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:58451 req_id = f2209474-487d-4873-a99a-a7c913c62491 response_code = 200 uri = /vpc-routes
21222024-11-18T18:45:19.717ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:48029
21232024-11-18T18:45:19.718ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
21242024-11-18T18:45:19.763ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
21252024-11-18T18:45:19.828ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 110076 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:48029 req_id = 6d67ac24-bc27-4fb6-90b8-6103ec321956 response_code = 204 uri = /switch-ports
21262024-11-18T18:45:19.828ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:49033
21272024-11-18T18:45:19.828ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:38996
21282024-11-18T18:45:19.828ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
21292024-11-18T18:45:19.828ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 247 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:58451 req_id = 54dfbd3b-3551-488c-8026-9ef2fc6587f5 response_code = 204 uri = /eip-gateways
21302024-11-18T18:45:19.828ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 338 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:48029 req_id = da3f5eff-9b71-4e22-a93f-ef270b816a3e response_code = 200 uri = /network-bootstore-config
21312024-11-18T18:45:19.828ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
21322024-11-18T18:45:19.832ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 4243 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:38996 req_id = 9d5c8e4b-4af5-4410-8695-cbeffbf2442c response_code = 200 uri = /v2p
21332024-11-18T18:45:19.838ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 249 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:58451 req_id = eeaa785c-4965-4a75-8f18-6719ea962e64 response_code = 204 uri = /vpc-routes
21342024-11-18T18:45:19.865ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
21352024-11-18T18:45:19.924ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 96023 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:49033 req_id = b6d2c118-8f52-45c5-b094-9e353e1cf3d8 response_code = 204 uri = /switch-ports
21362024-11-18T18:45:19.925ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 174 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:49033 req_id = 17dc325d-bb0b-4549-8342-6e6b091d013a response_code = 200 uri = /network-bootstore-config
21372024-11-18T18:45:32.299ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:39493
21382024-11-18T18:45:32.299ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 84 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:39493 req_id = ccd0a600-fdd7-45f9-bbef-50d71b5478db response_code = 200 uri = /baseboard
21392024-11-18T18:45:48.514ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:39576
21402024-11-18T18:45:48.514ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 75 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:39576 req_id = 6163d8e3-78b5-4302-bf04-1997290e6bbf response_code = 200 uri = /vpc-routes
21412024-11-18T18:45:48.601ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
21422024-11-18T18:45:48.601ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 101 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:39576 req_id = dbf91372-7c9f-4572-987b-f2135a79939e response_code = 204 uri = /eip-gateways
21432024-11-18T18:45:48.619ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 222 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:39576 req_id = ae0c107a-123c-46ff-a450-5aa03d1f249c response_code = 204 uri = /vpc-routes
21442024-11-18T18:45:48.760ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:53978
21452024-11-18T18:45:48.760ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 139 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:53978 req_id = fd09c0cf-ae4e-4ea4-b8bf-c785e1e83b12 response_code = 200 uri = /v2p
21462024-11-18T18:45:48.920ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:53656
21472024-11-18T18:45:48.920ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
21482024-11-18T18:45:48.962ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
21492024-11-18T18:45:49.024ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 103823 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:53656 req_id = 2ce8fa3e-1ae0-4653-bdde-cdec70093c93 response_code = 204 uri = /switch-ports
21502024-11-18T18:45:49.024ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 110 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:53656 req_id = 694f46b5-7075-4642-aa95-2f3e11bbefc1 response_code = 200 uri = /network-bootstore-config
21512024-11-18T18:45:49.053ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:36205
21522024-11-18T18:45:49.053ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:731 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 6a2febaf-2bbc-4cc0-aab6-80daf994ec33, kind: Service { id: 6767b855-63e1-4a7d-ac38-c4e7eb56747f }, name: Name("external-dns-6767b855-63e1-4a7d-ac38-c4e7eb56747f"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 152, 29])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 72590ac9-2812-4929-9b37-1829b4168efa, kind: Service { id: ceea2287-3571-41ea-8f15-b2aeb4a74fd9 }, name: Name("external-dns-ceea2287-3571-41ea-8f15-b2aeb4a74fd9"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 170, 19])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: ab6c5d8c-f7d0-4cf6-8c3c-5acf8f3959a9, kind: Service { id: d7ddf21e-0e1e-4004-b302-ecb81ba84c09 }, name: Name("nexus-d7ddf21e-0e1e-4004-b302-ecb81ba84c09"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 151, 123])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d0f6d114-ab1d-41fb-a2bb-c154f579e21f, kind: Service { id: 88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d }, name: Name("nexus-88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 37])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f6afceac-9bde-4010-ba9d-6c2eebc86b2d, kind: Service { id: 1bd1422d-700f-41e4-b299-7268d8edd3b1 }, name: Name("nexus-1bd1422d-700f-41e4-b299-7268d8edd3b1"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 0])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
21532024-11-18T18:45:49.053ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
21542024-11-18T18:45:49.054ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
21552024-11-18T18:45:49.054ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
21562024-11-18T18:45:49.054ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte0" rules = []
21572024-11-18T18:45:49.054ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
21582024-11-18T18:45:49.054ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
21592024-11-18T18:45:49.054ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 332 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:36205 req_id = 4029739c-c4b8-4790-8bf4-ac85cb93cd3b response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
21602024-11-18T18:45:49.633ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:36812
21612024-11-18T18:45:49.633ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 45 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:36812 req_id = e2eebaf6-65b3-4068-86fa-d3d8e45c4244 response_code = 200 uri = /vpc-routes
21622024-11-18T18:45:49.726ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:43472
21632024-11-18T18:45:49.727ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 129 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:43472 req_id = f815fd9d-cb11-4b29-9caf-a691cf95bc7b response_code = 200 uri = /v2p
21642024-11-18T18:45:49.808ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
21652024-11-18T18:45:49.808ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 72 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:36812 req_id = fe0b9265-4c5c-4e46-a4d9-40b68b55faf1 response_code = 204 uri = /eip-gateways
21662024-11-18T18:45:49.816ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 154 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:36812 req_id = 3e66fb8a-e7df-4273-a61c-05b90bcdbe6d response_code = 204 uri = /vpc-routes
21672024-11-18T18:45:49.852ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:44509
21682024-11-18T18:45:49.852ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 25 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:44509 req_id = e9484e67-fd2b-4807-b18f-623382ec1045 response_code = 200 uri = /vpc-routes
21692024-11-18T18:45:49.911ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:43973
21702024-11-18T18:45:49.911ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
21712024-11-18T18:45:49.955ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
21722024-11-18T18:45:50.016ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 104423 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:43973 req_id = be3db408-7e5f-49b8-945f-d6cf14b50439 response_code = 204 uri = /switch-ports
21732024-11-18T18:45:50.016ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:49369
21742024-11-18T18:45:50.016ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:52730
21752024-11-18T18:45:50.016ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:44470
21762024-11-18T18:45:50.016ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
21772024-11-18T18:45:50.016ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 282 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:44509 req_id = 0cddaa5c-eb71-4f2b-afdd-9732c552b8db response_code = 204 uri = /eip-gateways
21782024-11-18T18:45:50.016ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 354 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:43973 req_id = 635e0d33-09a0-4ec9-844f-534fd1643be9 response_code = 200 uri = /network-bootstore-config
21792024-11-18T18:45:50.016ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
21802024-11-18T18:45:50.016ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:731 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 6a2febaf-2bbc-4cc0-aab6-80daf994ec33, kind: Service { id: 6767b855-63e1-4a7d-ac38-c4e7eb56747f }, name: Name("external-dns-6767b855-63e1-4a7d-ac38-c4e7eb56747f"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 152, 29])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 72590ac9-2812-4929-9b37-1829b4168efa, kind: Service { id: ceea2287-3571-41ea-8f15-b2aeb4a74fd9 }, name: Name("external-dns-ceea2287-3571-41ea-8f15-b2aeb4a74fd9"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 170, 19])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: ab6c5d8c-f7d0-4cf6-8c3c-5acf8f3959a9, kind: Service { id: d7ddf21e-0e1e-4004-b302-ecb81ba84c09 }, name: Name("nexus-d7ddf21e-0e1e-4004-b302-ecb81ba84c09"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 151, 123])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d0f6d114-ab1d-41fb-a2bb-c154f579e21f, kind: Service { id: 88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d }, name: Name("nexus-88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 37])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f6afceac-9bde-4010-ba9d-6c2eebc86b2d, kind: Service { id: 1bd1422d-700f-41e4-b299-7268d8edd3b1 }, name: Name("nexus-1bd1422d-700f-41e4-b299-7268d8edd3b1"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 0])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
21812024-11-18T18:45:50.016ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 342 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:52730 req_id = c0d8f109-d2fd-4f11-9174-7e97a06fd818 response_code = 200 uri = /v2p
21822024-11-18T18:45:50.017ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
21832024-11-18T18:45:50.023ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
21842024-11-18T18:45:50.024ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
21852024-11-18T18:45:50.024ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte0" rules = []
21862024-11-18T18:45:50.024ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
21872024-11-18T18:45:50.024ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
21882024-11-18T18:45:50.024ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7568 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:44470 req_id = 54733082-cae3-48c9-a67f-ea0598f74c3f response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
21892024-11-18T18:45:50.029ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 227 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:44509 req_id = 0e3b10d8-c66b-42c8-b4c8-194b5686ffe7 response_code = 204 uri = /vpc-routes
21902024-11-18T18:45:50.056ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
21912024-11-18T18:45:50.109ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 93065 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:49369 req_id = 5b4aca36-6c68-4fe7-8a3a-a4e06fa39ae4 response_code = 204 uri = /switch-ports
21922024-11-18T18:45:50.110ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 161 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:49369 req_id = c6bbf771-7143-4165-8872-c3659f570714 response_code = 200 uri = /network-bootstore-config
21932024-11-18T18:45:50.246ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:41729
21942024-11-18T18:45:50.246ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:731 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 6a2febaf-2bbc-4cc0-aab6-80daf994ec33, kind: Service { id: 6767b855-63e1-4a7d-ac38-c4e7eb56747f }, name: Name("external-dns-6767b855-63e1-4a7d-ac38-c4e7eb56747f"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 152, 29])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 72590ac9-2812-4929-9b37-1829b4168efa, kind: Service { id: ceea2287-3571-41ea-8f15-b2aeb4a74fd9 }, name: Name("external-dns-ceea2287-3571-41ea-8f15-b2aeb4a74fd9"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 170, 19])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: ab6c5d8c-f7d0-4cf6-8c3c-5acf8f3959a9, kind: Service { id: d7ddf21e-0e1e-4004-b302-ecb81ba84c09 }, name: Name("nexus-d7ddf21e-0e1e-4004-b302-ecb81ba84c09"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 151, 123])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d0f6d114-ab1d-41fb-a2bb-c154f579e21f, kind: Service { id: 88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d }, name: Name("nexus-88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 37])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f6afceac-9bde-4010-ba9d-6c2eebc86b2d, kind: Service { id: 1bd1422d-700f-41e4-b299-7268d8edd3b1 }, name: Name("nexus-1bd1422d-700f-41e4-b299-7268d8edd3b1"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 0])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
21952024-11-18T18:45:50.246ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
21962024-11-18T18:45:50.246ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
21972024-11-18T18:45:50.246ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
21982024-11-18T18:45:50.246ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte0" rules = []
21992024-11-18T18:45:50.246ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
22002024-11-18T18:45:50.246ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
22012024-11-18T18:45:50.246ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 248 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:41729 req_id = afec5ac0-ee75-4a20-beca-0fa383fb1964 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
22022024-11-18T18:46:02.331ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:64205
22032024-11-18T18:46:02.331ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 67 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:64205 req_id = c619550e-8b2b-432f-969b-92fd382ee983 response_code = 200 uri = /baseboard
22042024-11-18T18:46:03.851ZINFOSledAgent (auto-cleanup-task): running automatic periodic zone bundle cleanup file = sled-agent/src/zone_bundle.rs:197
22052024-11-18T18:46:03.851ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
22062024-11-18T18:46:03.883ZINFOSledAgent (auto-cleanup-task): top of bundle cleanup loop file = sled-agent/src/zone_bundle.rs:186 next_cleanup = Instant { tv_sec: 1754, tv_nsec: 406229580 } time_to_next_cleanup = 599.99999591s
22072024-11-18T18:46:18.591ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:34424
22082024-11-18T18:46:18.591ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 69 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:34424 req_id = 2c61950d-de5c-4ea3-b547-ee8d761f4041 response_code = 200 uri = /vpc-routes
22092024-11-18T18:46:18.650ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
22102024-11-18T18:46:18.650ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 83 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:34424 req_id = 22b2d201-05a6-4d5e-a965-841aa849eb3a response_code = 204 uri = /eip-gateways
22112024-11-18T18:46:18.651ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
22122024-11-18T18:46:18.659ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 176 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:34424 req_id = 46fc41f5-fc71-4053-ba45-dbf451e3b2e8 response_code = 204 uri = /vpc-routes
22132024-11-18T18:46:18.728ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:45799
22142024-11-18T18:46:18.728ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 127 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:45799 req_id = ee1e9439-c74b-414b-b192-1664d03e12ac response_code = 200 uri = /v2p
22152024-11-18T18:46:18.776ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:61039
22162024-11-18T18:46:18.776ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
22172024-11-18T18:46:18.816ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
22182024-11-18T18:46:18.874ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 97742 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:61039 req_id = 76deff17-df21-43cf-9357-81bd6c0b7db4 response_code = 204 uri = /switch-ports
22192024-11-18T18:46:18.874ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 197 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:61039 req_id = 879593e4-ea61-4dda-aa0d-af84a2dcf617 response_code = 200 uri = /network-bootstore-config
22202024-11-18T18:46:19.537ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:45561
22212024-11-18T18:46:19.537ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 57 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:45561 req_id = 0a5a2573-28a3-4a47-b5c3-a498f6f9f937 response_code = 200 uri = /vpc-routes
22222024-11-18T18:46:19.625ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
22232024-11-18T18:46:19.625ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 74 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:45561 req_id = 321fabe5-a471-437c-a2a8-ca906b986f1f response_code = 204 uri = /eip-gateways
22242024-11-18T18:46:19.635ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 950 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:45561 req_id = 7041bc8e-5a27-48fb-976f-2f5f378c4891 response_code = 204 uri = /vpc-routes
22252024-11-18T18:46:19.668ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:42204
22262024-11-18T18:46:19.668ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 38 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:42204 req_id = 5f667644-f9c7-48a7-92b9-88478a0c7a9f response_code = 200 uri = /vpc-routes
22272024-11-18T18:46:19.731ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
22282024-11-18T18:46:19.731ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 58 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:42204 req_id = a18efa0a-b978-43e9-92b9-1f5abb98a192 response_code = 204 uri = /eip-gateways
22292024-11-18T18:46:19.733ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:55915
22302024-11-18T18:46:19.733ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 145 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:55915 req_id = c6bb977a-5e3d-43b1-b70a-2189500d36c7 response_code = 200 uri = /v2p
22312024-11-18T18:46:19.742ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 126 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:42204 req_id = aa7792b3-6188-470b-ac55-abcb907af848 response_code = 204 uri = /vpc-routes
22322024-11-18T18:46:19.751ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51819
22332024-11-18T18:46:19.751ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 88 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:51819 req_id = e535cbd9-ea4c-4b2a-a612-91a353224bcc response_code = 200 uri = /v2p
22342024-11-18T18:46:19.753ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:40137
22352024-11-18T18:46:19.753ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
22362024-11-18T18:46:19.791ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
22372024-11-18T18:46:19.850ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 96928 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:40137 req_id = 47e29799-aa75-41c9-ac48-91e072d1831a response_code = 204 uri = /switch-ports
22382024-11-18T18:46:19.853ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 82 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:40137 req_id = 4c2f776c-0e96-429b-b46a-dbda23accb95 response_code = 200 uri = /network-bootstore-config
22392024-11-18T18:46:19.870ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:41427
22402024-11-18T18:46:19.870ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
22412024-11-18T18:46:19.907ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
22422024-11-18T18:46:19.965ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 94743 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:41427 req_id = f8d8d9d6-1cf9-450d-930c-5da1985da9a0 response_code = 204 uri = /switch-ports
22432024-11-18T18:46:19.965ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 171 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:41427 req_id = 990745a4-5f1f-424e-9c3d-ca0139d318b8 response_code = 200 uri = /network-bootstore-config
22442024-11-18T18:46:32.364ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:62649
22452024-11-18T18:46:32.364ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 55 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:62649 req_id = 5168f4a0-66e2-418a-9e90-9661f5367ab3 response_code = 200 uri = /baseboard
22462024-11-18T18:46:48.737ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:63376
22472024-11-18T18:46:48.737ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 92 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:63376 req_id = b55368e4-777f-4d79-8396-d63d18be01c8 response_code = 200 uri = /vpc-routes
22482024-11-18T18:46:48.759ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
22492024-11-18T18:46:48.759ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 123 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:63376 req_id = e6a1826f-ee0c-4d03-b503-47981f100e03 response_code = 204 uri = /eip-gateways
22502024-11-18T18:46:48.771ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 196 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:63376 req_id = a94859fc-732b-47a8-83dd-e242be034fbe response_code = 204 uri = /vpc-routes
22512024-11-18T18:46:48.932ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:42369
22522024-11-18T18:46:48.932ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 122 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:42369 req_id = 355319f7-d24e-47eb-83b7-ab227e0fa42a response_code = 200 uri = /v2p
22532024-11-18T18:46:49.034ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:44648
22542024-11-18T18:46:49.034ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
22552024-11-18T18:46:49.076ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
22562024-11-18T18:46:49.140ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 106450 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:44648 req_id = ef373b41-6e24-4db6-be13-15ed6ca1d6c7 response_code = 204 uri = /switch-ports
22572024-11-18T18:46:49.141ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 118 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:44648 req_id = 47c885e3-4beb-4062-81fb-f0b538ed144a response_code = 200 uri = /network-bootstore-config
22582024-11-18T18:46:49.705ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:52635
22592024-11-18T18:46:49.705ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 41 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:52635 req_id = 5853fbab-31d1-4019-8672-e53132ab6990 response_code = 200 uri = /vpc-routes
22602024-11-18T18:46:49.721ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
22612024-11-18T18:46:49.721ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 52 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:52635 req_id = 52ddb541-9cb2-4199-bb3a-c8b577ec25e5 response_code = 204 uri = /eip-gateways
22622024-11-18T18:46:49.729ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 168 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:52635 req_id = 39179ebc-c5a8-4847-a5b9-b0ba580e1be9 response_code = 204 uri = /vpc-routes
22632024-11-18T18:46:49.797ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:55618
22642024-11-18T18:46:49.797ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 69 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:55618 req_id = a920719f-1030-418b-8ebe-50f25d95cb38 response_code = 200 uri = /vpc-routes
22652024-11-18T18:46:49.842ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:33042
22662024-11-18T18:46:49.842ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 181 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:33042 req_id = 9814ab9b-1cc0-465e-b5a8-e3565955a598 response_code = 200 uri = /v2p
22672024-11-18T18:46:49.857ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
22682024-11-18T18:46:49.857ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 56 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:55618 req_id = bac09d25-b430-4e69-9b71-17aa064fa981 response_code = 204 uri = /eip-gateways
22692024-11-18T18:46:49.899ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:45026
22702024-11-18T18:46:49.899ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
22712024-11-18T18:46:49.941ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
22722024-11-18T18:46:50.004ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 105709 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:45026 req_id = eb2a38c2-dd60-405b-8959-1fa65db24c57 response_code = 204 uri = /switch-ports
22732024-11-18T18:46:50.005ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:56105
22742024-11-18T18:46:50.005ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 346 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:55618 req_id = 420a5d7f-0348-4f80-9f92-e01878ffc1ff response_code = 204 uri = /vpc-routes
22752024-11-18T18:46:50.005ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 302 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:45026 req_id = b58f39c7-5241-4cac-a103-9fb1361b3f0f response_code = 200 uri = /network-bootstore-config
22762024-11-18T18:46:50.005ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 269 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:56105 req_id = ecc2bfa6-b480-40f3-a574-5af4f7d114b0 response_code = 200 uri = /v2p
22772024-11-18T18:46:50.125ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:42881
22782024-11-18T18:46:50.125ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
22792024-11-18T18:46:50.170ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
22802024-11-18T18:46:50.230ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 104802 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:42881 req_id = 3008f3ed-26a9-471b-9f6e-93d042c1de73 response_code = 204 uri = /switch-ports
22812024-11-18T18:46:50.233ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 91 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:42881 req_id = 572143ee-702e-4616-97d5-04c7ca829acf response_code = 200 uri = /network-bootstore-config
22822024-11-18T18:47:02.396ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:60052
22832024-11-18T18:47:02.396ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 64 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:60052 req_id = 797f1f21-58a3-4bd2-8677-82df1a6b180d response_code = 200 uri = /baseboard
22842024-11-18T18:47:18.549ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:52303
22852024-11-18T18:47:18.549ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 53 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:52303 req_id = d2cfa65b-68f0-4eb6-aa09-df1140ffd660 response_code = 200 uri = /vpc-routes
22862024-11-18T18:47:18.611ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
22872024-11-18T18:47:18.611ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 105 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:52303 req_id = 630c62a5-10b9-4cea-9d5b-d3a90ce9300c response_code = 204 uri = /eip-gateways
22882024-11-18T18:47:18.621ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 211 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:52303 req_id = f9dd4acc-814e-4e97-a5cc-ef8e707b0041 response_code = 204 uri = /vpc-routes
22892024-11-18T18:47:18.727ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:62250
22902024-11-18T18:47:18.728ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 143 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:62250 req_id = fa8fb231-b4fb-4668-996c-54788745f928 response_code = 200 uri = /v2p
22912024-11-18T18:47:18.753ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:48282
22922024-11-18T18:47:18.753ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
22932024-11-18T18:47:18.793ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
22942024-11-18T18:47:18.856ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 103547 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:48282 req_id = 40ba6027-bc82-4ad6-b834-c1354d3408e1 response_code = 204 uri = /switch-ports
22952024-11-18T18:47:18.856ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 91 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:48282 req_id = 44207ee2-e14d-4a61-a26b-adc157b3fd6e response_code = 200 uri = /network-bootstore-config
22962024-11-18T18:47:19.551ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:38737
22972024-11-18T18:47:19.551ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 57 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:38737 req_id = bf7f48f3-349a-47d5-8cc7-f6eb5427ba6c response_code = 200 uri = /vpc-routes
22982024-11-18T18:47:19.566ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:52915
22992024-11-18T18:47:19.566ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 35 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:52915 req_id = f0a42daa-7327-461f-beaa-a75ddccba166 response_code = 200 uri = /vpc-routes
23002024-11-18T18:47:19.586ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
23012024-11-18T18:47:19.587ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 101 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:52915 req_id = 3bad2d6b-f431-44f0-b5c0-cb56e806f89f response_code = 204 uri = /eip-gateways
23022024-11-18T18:47:19.597ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 218 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:52915 req_id = 9986c622-36df-4bc4-9dd1-5d636dbce2b4 response_code = 204 uri = /vpc-routes
23032024-11-18T18:47:19.608ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:48129
23042024-11-18T18:47:19.608ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
23052024-11-18T18:47:19.652ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
23062024-11-18T18:47:19.725ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 116770 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:48129 req_id = 1d4b381a-b30a-4446-a72a-36b968a93558 response_code = 204 uri = /switch-ports
23072024-11-18T18:47:19.726ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
23082024-11-18T18:47:19.726ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 172 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:38737 req_id = fdbd005f-a716-43c1-8ab1-bf503363b9d5 response_code = 204 uri = /eip-gateways
23092024-11-18T18:47:19.726ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 259 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:48129 req_id = ab9c1893-ddaf-4318-bece-f24e0720dec9 response_code = 200 uri = /network-bootstore-config
23102024-11-18T18:47:19.730ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:52110
23112024-11-18T18:47:19.730ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
23122024-11-18T18:47:19.771ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
23132024-11-18T18:47:19.834ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 103872 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:52110 req_id = db841510-9363-471f-8e29-4e8c0def6d7a response_code = 204 uri = /switch-ports
23142024-11-18T18:47:19.834ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:64697
23152024-11-18T18:47:19.834ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 174 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:52110 req_id = 2e4f806c-afc2-4f3c-b989-de5328a9880e response_code = 200 uri = /network-bootstore-config
23162024-11-18T18:47:19.834ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 360 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:38737 req_id = bab84485-7bf0-404b-8fbc-a42cfdf90394 response_code = 204 uri = /vpc-routes
23172024-11-18T18:47:19.834ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 261 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:64697 req_id = 92912db1-ebd8-4197-a373-d4234a80680d response_code = 200 uri = /v2p
23182024-11-18T18:47:19.841ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:57219
23192024-11-18T18:47:19.841ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 155 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:57219 req_id = 0d54456d-91fc-420c-b731-e3875e9ff5f0 response_code = 200 uri = /v2p
23202024-11-18T18:47:32.430ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:62045
23212024-11-18T18:47:32.430ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 63 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:62045 req_id = 02d040d1-9c03-4ece-864c-b415e564b8b1 response_code = 200 uri = /baseboard
23222024-11-18T18:47:48.716ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:47687
23232024-11-18T18:47:48.716ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 92 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:47687 req_id = baaa0950-cd39-4c75-9437-afe75aa0d8b0 response_code = 200 uri = /vpc-routes
23242024-11-18T18:47:48.781ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
23252024-11-18T18:47:48.781ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 83 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:47687 req_id = 8dbd157f-2957-41d4-994e-a0878c5ce2d0 response_code = 204 uri = /eip-gateways
23262024-11-18T18:47:48.791ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 188 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:47687 req_id = 2abb113c-1581-4ea5-ad0d-94da054a40af response_code = 204 uri = /vpc-routes
23272024-11-18T18:47:48.964ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:56588
23282024-11-18T18:47:48.964ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 107 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:56588 req_id = 08cdaf96-043b-400a-a1f5-934c68a802ae response_code = 200 uri = /v2p
23292024-11-18T18:47:48.988ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:33228
23302024-11-18T18:47:48.988ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
23312024-11-18T18:47:49.027ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
23322024-11-18T18:47:49.080ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 91917 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:33228 req_id = 4e6928be-a28f-4a4d-9156-ee59f2c109bd response_code = 204 uri = /switch-ports
23332024-11-18T18:47:49.080ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 105 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:33228 req_id = d017fcf5-3afb-4d2b-8ec9-95141bd09f06 response_code = 200 uri = /network-bootstore-config
23342024-11-18T18:47:49.679ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:40100
23352024-11-18T18:47:49.679ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 59 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:40100 req_id = 4a9b97c3-6473-4286-a506-9444a41d3295 response_code = 200 uri = /vpc-routes
23362024-11-18T18:47:49.699ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
23372024-11-18T18:47:49.699ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 62 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:40100 req_id = 371f4e53-a28a-47b8-9309-d6b163b76eef response_code = 204 uri = /eip-gateways
23382024-11-18T18:47:49.707ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 146 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:40100 req_id = 29fb9164-61d1-4f8f-94ea-cbd472fb532d response_code = 204 uri = /vpc-routes
23392024-11-18T18:47:49.767ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:34076
23402024-11-18T18:47:49.768ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 115 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:34076 req_id = d645fc6f-6bf5-45c8-8e9a-06ad52426773 response_code = 200 uri = /v2p
23412024-11-18T18:47:49.780ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:42745
23422024-11-18T18:47:49.780ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 21 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:42745 req_id = 7f628a79-f07e-467f-9b34-ec75a2830d5f response_code = 200 uri = /vpc-routes
23432024-11-18T18:47:49.829ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:41930
23442024-11-18T18:47:49.830ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
23452024-11-18T18:47:49.873ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
23462024-11-18T18:47:49.934ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 104761 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:41930 req_id = ab72aadd-0106-4850-8bf0-07dac7cd6c80 response_code = 204 uri = /switch-ports
23472024-11-18T18:47:49.935ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:47297
23482024-11-18T18:47:49.935ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
23492024-11-18T18:47:49.935ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 177 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:42745 req_id = eb673462-8ec8-4dcf-afb0-824a398d89ab response_code = 204 uri = /eip-gateways
23502024-11-18T18:47:49.935ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 233 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:41930 req_id = 5e418838-8826-43eb-8176-6d73fda3a8da response_code = 200 uri = /network-bootstore-config
23512024-11-18T18:47:49.935ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 282 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:47297 req_id = 88338c28-b835-4600-a3f3-8dc3d69e75fb response_code = 200 uri = /v2p
23522024-11-18T18:47:49.945ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 239 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:42745 req_id = a708d4a8-63f6-4198-abf7-2b2720f79e13 response_code = 204 uri = /vpc-routes
23532024-11-18T18:47:49.951ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:54882
23542024-11-18T18:47:49.951ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
23552024-11-18T18:47:49.989ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
23562024-11-18T18:47:50.049ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 98054 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:54882 req_id = cc7ed2f9-106f-4234-8987-207793e9d900 response_code = 204 uri = /switch-ports
23572024-11-18T18:47:50.049ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 115 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:54882 req_id = 56afc4f6-1476-4138-9755-52c5e30ee87f response_code = 200 uri = /network-bootstore-config
23582024-11-18T18:48:02.464ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:54890
23592024-11-18T18:48:02.464ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 73 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:54890 req_id = ac7834c0-819d-47f9-86f6-5034dbf505fa response_code = 200 uri = /baseboard
23602024-11-18T18:48:15.860ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 498 n_new_samples = 6 n_overflow_samples = 4 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
23612024-11-18T18:48:15.909ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 498 n_new_samples = 6 n_overflow_samples = 4 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
23622024-11-18T18:48:18.476ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:43680
23632024-11-18T18:48:18.477ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 102 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:43680 req_id = a3e39a92-5512-4e4f-9eee-4f5bb0fdbfe0 response_code = 200 uri = /vpc-routes
23642024-11-18T18:48:18.554ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
23652024-11-18T18:48:18.554ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 77 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:43680 req_id = 426068d3-1665-473a-b3ef-dad7358277be response_code = 204 uri = /eip-gateways
23662024-11-18T18:48:18.564ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 180 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:43680 req_id = 55b35bce-a7b6-46ea-bed6-caa79d660e41 response_code = 204 uri = /vpc-routes
23672024-11-18T18:48:18.622ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:43072
23682024-11-18T18:48:18.623ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 122 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:43072 req_id = 64ee5a41-55b8-4388-a45a-45b9b167a74b response_code = 200 uri = /v2p
23692024-11-18T18:48:18.635ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:57196
23702024-11-18T18:48:18.635ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
23712024-11-18T18:48:18.673ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
23722024-11-18T18:48:18.729ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 94669 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:57196 req_id = f0c4aa8a-5186-402c-a059-30e0aeefb9cf response_code = 204 uri = /switch-ports
23732024-11-18T18:48:18.730ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 90 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:57196 req_id = 51eead00-2b96-44eb-9c0d-bd4f967d7d1b response_code = 200 uri = /network-bootstore-config
23742024-11-18T18:48:19.532ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:54321
23752024-11-18T18:48:19.532ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 125 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:54321 req_id = a21c187c-605d-4470-90fa-048454540f07 response_code = 200 uri = /vpc-routes
23762024-11-18T18:48:19.612ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
23772024-11-18T18:48:19.612ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 90 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:54321 req_id = a2f1d003-ff31-45f7-8b8b-0cfa422a9f7b response_code = 204 uri = /eip-gateways
23782024-11-18T18:48:19.622ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 203 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:54321 req_id = b92ee75e-21c7-4adc-b9a2-51537e971726 response_code = 204 uri = /vpc-routes
23792024-11-18T18:48:19.649ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:33153
23802024-11-18T18:48:19.650ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 126 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:33153 req_id = 37244523-4a7c-4e09-8c93-f40afb24d057 response_code = 200 uri = /v2p
23812024-11-18T18:48:19.698ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:54480
23822024-11-18T18:48:19.698ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
23832024-11-18T18:48:19.740ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
23842024-11-18T18:48:19.801ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 102944 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:54480 req_id = e07fcb10-bc1c-4869-a6dc-d087a0eb40c6 response_code = 204 uri = /switch-ports
23852024-11-18T18:48:19.801ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:40924
23862024-11-18T18:48:19.801ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:44158
23872024-11-18T18:48:19.801ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:41605
23882024-11-18T18:48:19.801ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 278 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:54480 req_id = 93c3c669-8b8b-4d87-bdbc-e374a95b889b response_code = 200 uri = /network-bootstore-config
23892024-11-18T18:48:19.801ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
23902024-11-18T18:48:19.801ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 125 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:44158 req_id = 1ace782d-495d-4abb-aa8b-fc70e3dbd422 response_code = 200 uri = /vpc-routes
23912024-11-18T18:48:19.801ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 247 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:40924 req_id = 481b430d-703a-4c02-b007-9618193c8b41 response_code = 200 uri = /v2p
23922024-11-18T18:48:19.840ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
23932024-11-18T18:48:19.857ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
23942024-11-18T18:48:19.857ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 153 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:44158 req_id = 07c7760d-9770-4b98-ada9-ba7771ec5d75 response_code = 204 uri = /eip-gateways
23952024-11-18T18:48:19.867ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 966 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:44158 req_id = b005e64d-114c-49db-8c26-17a1b0751f25 response_code = 204 uri = /vpc-routes
23962024-11-18T18:48:19.899ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 97443 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:41605 req_id = d6bb4fd5-f888-4eee-822c-98b40da15c60 response_code = 204 uri = /switch-ports
23972024-11-18T18:48:19.899ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 137 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:41605 req_id = 6e687aae-8bbe-4e84-bc8f-2ea025439069 response_code = 200 uri = /network-bootstore-config
23982024-11-18T18:48:25.857ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
23992024-11-18T18:48:25.859ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
24002024-11-18T18:48:25.862ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
24012024-11-18T18:48:25.911ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
24022024-11-18T18:48:32.496ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:61126
24032024-11-18T18:48:32.496ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 65 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:61126 req_id = da1f1114-cc7c-430a-81b1-44314c0ff5e3 response_code = 200 uri = /baseboard
24042024-11-18T18:48:35.859ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
24052024-11-18T18:48:35.861ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
24062024-11-18T18:48:35.864ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
24072024-11-18T18:48:35.912ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
24082024-11-18T18:48:45.861ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
24092024-11-18T18:48:45.862ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
24102024-11-18T18:48:45.865ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
24112024-11-18T18:48:45.915ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
24122024-11-18T18:48:48.725ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:45092
24132024-11-18T18:48:48.725ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 99 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:45092 req_id = 136c3071-c108-432b-b457-cadaaf418d4e response_code = 200 uri = /vpc-routes
24142024-11-18T18:48:48.788ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
24152024-11-18T18:48:48.788ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 97 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:45092 req_id = f63015a6-261b-41b5-9356-604dca6f004d response_code = 204 uri = /eip-gateways
24162024-11-18T18:48:48.798ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 196 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:45092 req_id = bc5cf6ad-b895-4344-a547-053ac10fe1b2 response_code = 204 uri = /vpc-routes
24172024-11-18T18:48:48.921ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:63683
24182024-11-18T18:48:48.922ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
24192024-11-18T18:48:48.963ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
24202024-11-18T18:48:49.027ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 105165 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:63683 req_id = ee031001-c844-4cc0-a11b-8ac70039397e response_code = 204 uri = /switch-ports
24212024-11-18T18:48:49.027ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:42518
24222024-11-18T18:48:49.027ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 284 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:63683 req_id = 61552843-e823-486a-ad84-29b177e50849 response_code = 200 uri = /network-bootstore-config
24232024-11-18T18:48:49.027ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 214 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:42518 req_id = 5160f211-843f-4cd4-a064-00ea31d7c23d response_code = 200 uri = /v2p
24242024-11-18T18:48:49.556ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:48267
24252024-11-18T18:48:49.556ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 62 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:48267 req_id = b4d82541-5467-4ae9-8c0f-b5be91bb540f response_code = 200 uri = /vpc-routes
24262024-11-18T18:48:49.664ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
24272024-11-18T18:48:49.664ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 83 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:48267 req_id = da4152a5-3e03-4627-9f10-bb3a3b8e8665 response_code = 204 uri = /eip-gateways
24282024-11-18T18:48:49.732ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:54591
24292024-11-18T18:48:49.732ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 133 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:54591 req_id = 73d81a40-7155-4977-b5cf-bcedd773fa96 response_code = 200 uri = /v2p
24302024-11-18T18:48:49.746ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 230 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:48267 req_id = 1f203f40-6b28-4580-87a3-07b1f5ea7c8c response_code = 204 uri = /vpc-routes
24312024-11-18T18:48:49.813ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:52819
24322024-11-18T18:48:49.813ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
24332024-11-18T18:48:49.849ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
24342024-11-18T18:48:49.917ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 104407 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:52819 req_id = 28312a7c-a25d-4c58-8969-fee5eda6e07e response_code = 204 uri = /switch-ports
24352024-11-18T18:48:49.917ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:40318
24362024-11-18T18:48:49.918ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 119 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:40318 req_id = bd0a2959-842f-4fda-a781-9ceb1a671f06 response_code = 200 uri = /vpc-routes
24372024-11-18T18:48:49.918ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 241 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:52819 req_id = f0339de9-12ea-45ac-82de-55b4a0c13e39 response_code = 200 uri = /network-bootstore-config
24382024-11-18T18:48:49.988ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
24392024-11-18T18:48:49.988ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 79 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:40318 req_id = a908b702-d4b3-488c-82cb-769d203075f5 response_code = 204 uri = /eip-gateways
24402024-11-18T18:48:50.001ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 152 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:40318 req_id = 995588a0-553e-4bfc-a536-df29054dd3c5 response_code = 204 uri = /vpc-routes
24412024-11-18T18:48:50.006ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:43673
24422024-11-18T18:48:50.006ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
24432024-11-18T18:48:50.047ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
24442024-11-18T18:48:50.100ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 94510 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:43673 req_id = ade1b67f-a9ea-4edc-8c9b-33398614bd2d response_code = 204 uri = /switch-ports
24452024-11-18T18:48:50.100ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:46839
24462024-11-18T18:48:50.101ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 247 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:43673 req_id = 4451229f-8153-4d74-aee6-964bf7df6826 response_code = 200 uri = /network-bootstore-config
24472024-11-18T18:48:50.101ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 298 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:46839 req_id = acec1816-6aac-4db4-a7ef-3e69c234afdc response_code = 200 uri = /v2p
24482024-11-18T18:48:55.864ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
24492024-11-18T18:48:55.864ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
24502024-11-18T18:48:55.868ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
24512024-11-18T18:48:55.917ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
24522024-11-18T18:49:02.529ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:45300
24532024-11-18T18:49:02.529ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 79 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:45300 req_id = e09eb9cc-6bb3-4ef1-afd8-f11212706637 response_code = 200 uri = /baseboard
24542024-11-18T18:49:05.866ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
24552024-11-18T18:49:05.866ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
24562024-11-18T18:49:05.869ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
24572024-11-18T18:49:05.919ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
24582024-11-18T18:49:15.867ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
24592024-11-18T18:49:15.868ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
24602024-11-18T18:49:15.871ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
24612024-11-18T18:49:15.921ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
24622024-11-18T18:49:18.648ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:41484
24632024-11-18T18:49:18.648ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 84 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:41484 req_id = 4dde73e6-99a8-46b5-a30d-1c295d740580 response_code = 200 uri = /vpc-routes
24642024-11-18T18:49:18.706ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
24652024-11-18T18:49:18.706ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 74 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:41484 req_id = f52b53ae-0c2d-46a0-b0e6-95217d374ea8 response_code = 204 uri = /eip-gateways
24662024-11-18T18:49:18.715ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 193 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:41484 req_id = fd79a5b5-21b2-4146-a4a6-0c4af583d4c0 response_code = 204 uri = /vpc-routes
24672024-11-18T18:49:18.757ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:43577
24682024-11-18T18:49:18.757ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
24692024-11-18T18:49:18.798ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
24702024-11-18T18:49:18.859ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 102091 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:43577 req_id = 34751f43-5632-4a3e-b003-1a527922d425 response_code = 204 uri = /switch-ports
24712024-11-18T18:49:18.859ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:38160
24722024-11-18T18:49:18.860ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 282 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:43577 req_id = 750d9b97-e3c4-4543-8d7c-bb1de22413cc response_code = 200 uri = /network-bootstore-config
24732024-11-18T18:49:18.860ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 213 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:38160 req_id = 0a7bc2da-df43-486b-b7d5-a63b28ca2511 response_code = 200 uri = /v2p
24742024-11-18T18:49:19.534ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:61750
24752024-11-18T18:49:19.534ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 42 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:61750 req_id = 7d54167d-a94e-4771-befc-8c714180a580 response_code = 200 uri = /vpc-routes
24762024-11-18T18:49:19.571ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
24772024-11-18T18:49:19.571ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 75 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:61750 req_id = 466e8bfe-f072-469e-a112-94bad9d5ff12 response_code = 204 uri = /eip-gateways
24782024-11-18T18:49:19.581ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 197 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:61750 req_id = f14b16bf-b571-41d6-ac20-e08698303ff5 response_code = 204 uri = /vpc-routes
24792024-11-18T18:49:19.604ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51895
24802024-11-18T18:49:19.604ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
24812024-11-18T18:49:19.654ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
24822024-11-18T18:49:19.729ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 125408 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:51895 req_id = 83e58499-e850-4497-aca3-2d21ddf7b7d7 response_code = 204 uri = /switch-ports
24832024-11-18T18:49:19.730ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:63903
24842024-11-18T18:49:19.730ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:56255
24852024-11-18T18:49:19.730ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:62989
24862024-11-18T18:49:19.730ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
24872024-11-18T18:49:19.730ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 290 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:51895 req_id = d6614eef-4a85-400c-a6ca-41b7e5193cea response_code = 200 uri = /network-bootstore-config
24882024-11-18T18:49:19.730ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 145 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:56255 req_id = 65bfcdb5-410a-4663-bfae-fd8906b1233c response_code = 200 uri = /vpc-routes
24892024-11-18T18:49:19.735ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 5391 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:63903 req_id = 46675f44-ad1e-4a47-8872-8f951b39d620 response_code = 200 uri = /v2p
24902024-11-18T18:49:19.774ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
24912024-11-18T18:49:19.792ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
24922024-11-18T18:49:19.792ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 572 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:56255 req_id = 7d59e3d4-91cd-4b55-a962-ff8218f462ac response_code = 204 uri = /eip-gateways
24932024-11-18T18:49:19.807ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1015 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:56255 req_id = 64005a71-d7d0-4fee-9c24-8260412cce06 response_code = 204 uri = /vpc-routes
24942024-11-18T18:49:19.852ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 122550 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:62989 req_id = fdcbafac-5910-4cee-8d84-b38a1e427d52 response_code = 204 uri = /switch-ports
24952024-11-18T18:49:19.853ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 203 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:62989 req_id = 4fb412a8-410c-4728-8583-6447e3365e19 response_code = 200 uri = /network-bootstore-config
24962024-11-18T18:49:19.966ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:38028
24972024-11-18T18:49:19.966ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 125 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:38028 req_id = b65a816d-3c2e-464f-8c7c-1c1a9ac98470 response_code = 200 uri = /v2p
24982024-11-18T18:49:25.869ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
24992024-11-18T18:49:25.870ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
25002024-11-18T18:49:25.873ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
25012024-11-18T18:49:25.923ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
25022024-11-18T18:49:32.562ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:35784
25032024-11-18T18:49:32.562ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 68 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:35784 req_id = 179e0610-9a2a-4053-bd01-a6465f594d30 response_code = 200 uri = /baseboard
25042024-11-18T18:49:35.872ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
25052024-11-18T18:49:35.872ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
25062024-11-18T18:49:35.874ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
25072024-11-18T18:49:35.924ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
25082024-11-18T18:49:45.874ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
25092024-11-18T18:49:45.875ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
25102024-11-18T18:49:45.875ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
25112024-11-18T18:49:45.926ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
25122024-11-18T18:49:48.772ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:37329
25132024-11-18T18:49:48.772ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 73 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:37329 req_id = 8b6f38a5-3db7-4c98-b185-f9e6e8eb7ecb response_code = 200 uri = /vpc-routes
25142024-11-18T18:49:48.839ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
25152024-11-18T18:49:48.839ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 87 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:37329 req_id = 016ae31d-ef5a-428f-9a18-a4517e4664f2 response_code = 204 uri = /eip-gateways
25162024-11-18T18:49:48.849ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 198 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:37329 req_id = fe6481a1-5446-4239-a368-132a86d1a02c response_code = 204 uri = /vpc-routes
25172024-11-18T18:49:48.909ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:55036
25182024-11-18T18:49:48.910ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 160 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:55036 req_id = 7a3675d1-39eb-4940-94e7-9a791319c318 response_code = 200 uri = /v2p
25192024-11-18T18:49:49.029ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:37682
25202024-11-18T18:49:49.029ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
25212024-11-18T18:49:49.073ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
25222024-11-18T18:49:49.153ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 124729 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:37682 req_id = 6b0cea23-0cac-421d-be90-ab647da8be9d response_code = 204 uri = /switch-ports
25232024-11-18T18:49:49.159ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 97 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:37682 req_id = 79574694-6a1b-4ec9-abac-1c08ec717300 response_code = 200 uri = /network-bootstore-config
25242024-11-18T18:49:49.597ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:41403
25252024-11-18T18:49:49.597ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 114 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:41403 req_id = 969fb513-a0d2-4837-a680-9e92c7a69f62 response_code = 200 uri = /vpc-routes
25262024-11-18T18:49:49.616ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
25272024-11-18T18:49:49.616ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 108 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:41403 req_id = 17863686-00d4-457b-8c38-7669b28ba378 response_code = 204 uri = /eip-gateways
25282024-11-18T18:49:49.631ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 257 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:41403 req_id = 1ca441d9-0949-46da-b79e-63a62a010846 response_code = 204 uri = /vpc-routes
25292024-11-18T18:49:49.697ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:54639
25302024-11-18T18:49:49.697ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 180 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:54639 req_id = 07f42016-00af-47a8-a3ea-cc46a8fe1c94 response_code = 200 uri = /v2p
25312024-11-18T18:49:49.799ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:47659
25322024-11-18T18:49:49.799ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
25332024-11-18T18:49:49.843ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
25342024-11-18T18:49:49.899ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 99857 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:47659 req_id = 7f46c7ac-4c27-4a52-8016-4dcff2b1f4d6 response_code = 204 uri = /switch-ports
25352024-11-18T18:49:49.899ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:47876
25362024-11-18T18:49:49.899ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 47 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:47876 req_id = 23d75e5d-ad2c-4d4a-b44b-903c9bb2a215 response_code = 200 uri = /vpc-routes
25372024-11-18T18:49:49.899ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 86 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:47659 req_id = 29897feb-1597-4a32-b1bf-3ede29e10f48 response_code = 200 uri = /network-bootstore-config
25382024-11-18T18:49:49.913ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:50817
25392024-11-18T18:49:49.914ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
25402024-11-18T18:49:49.956ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
25412024-11-18T18:49:50.019ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 105096 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:50817 req_id = b7ed155b-f0f8-4dc3-856a-3d54a5d8f760 response_code = 204 uri = /switch-ports
25422024-11-18T18:49:50.019ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:62757
25432024-11-18T18:49:50.019ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
25442024-11-18T18:49:50.019ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 164 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:47876 req_id = 4c971bb2-9710-400e-9c2f-e5c50434c7b7 response_code = 204 uri = /eip-gateways
25452024-11-18T18:49:50.019ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 192 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:50817 req_id = edf9c9b8-a519-45f2-aacf-5c3d35447b31 response_code = 200 uri = /network-bootstore-config
25462024-11-18T18:49:50.019ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 266 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:62757 req_id = e08c98e8-b113-4e0e-920b-07d0efa0a7d9 response_code = 200 uri = /v2p
25472024-11-18T18:49:50.029ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 245 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:47876 req_id = f8f48530-af0b-4cff-bab4-c18c8a440d29 response_code = 204 uri = /vpc-routes
25482024-11-18T18:49:55.876ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
25492024-11-18T18:49:55.876ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
25502024-11-18T18:49:55.877ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
25512024-11-18T18:49:55.928ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
25522024-11-18T18:50:02.595ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:40941
25532024-11-18T18:50:02.596ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 92 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:40941 req_id = 836c2684-8333-44b0-8d92-c41f24545d66 response_code = 200 uri = /baseboard
25542024-11-18T18:50:05.877ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
25552024-11-18T18:50:05.879ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
25562024-11-18T18:50:05.879ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
25572024-11-18T18:50:05.931ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
25582024-11-18T18:50:15.879ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
25592024-11-18T18:50:15.880ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
25602024-11-18T18:50:15.881ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
25612024-11-18T18:50:15.933ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
25622024-11-18T18:50:18.581ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:32846
25632024-11-18T18:50:18.581ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 83 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:32846 req_id = bc983c01-0eb4-4c93-a44f-076cd0360f39 response_code = 200 uri = /vpc-routes
25642024-11-18T18:50:18.649ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
25652024-11-18T18:50:18.649ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 67 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:32846 req_id = 2ae6e649-17f3-4fed-9e90-9b55a8a63296 response_code = 204 uri = /eip-gateways
25662024-11-18T18:50:18.658ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 160 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:32846 req_id = 0c4e7914-f654-419e-881d-5f6047ffedb8 response_code = 204 uri = /vpc-routes
25672024-11-18T18:50:18.696ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:54647
25682024-11-18T18:50:18.697ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
25692024-11-18T18:50:18.738ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
25702024-11-18T18:50:18.800ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 103138 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:54647 req_id = 914a1f17-1092-4f59-886e-64d7981c9d0e response_code = 204 uri = /switch-ports
25712024-11-18T18:50:18.800ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:46256
25722024-11-18T18:50:18.800ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 216 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:46256 req_id = 2154321e-5d19-493c-a161-88e93219a772 response_code = 200 uri = /v2p
25732024-11-18T18:50:18.800ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 226 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:54647 req_id = c519186c-7269-4408-8ff1-57c295499bf2 response_code = 200 uri = /network-bootstore-config
25742024-11-18T18:50:19.542ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:40888
25752024-11-18T18:50:19.542ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 60 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:40888 req_id = e3e1f4d8-8484-41d4-8a30-ec8607859bf2 response_code = 200 uri = /vpc-routes
25762024-11-18T18:50:19.636ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
25772024-11-18T18:50:19.636ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 72 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:40888 req_id = 91c5c413-2316-44f0-bdd9-b6b7ed02c9df response_code = 204 uri = /eip-gateways
25782024-11-18T18:50:19.647ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 157 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:40888 req_id = 9bf2c55e-a8a3-412f-afd6-bb2737987f4d response_code = 204 uri = /vpc-routes
25792024-11-18T18:50:19.676ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:47274
25802024-11-18T18:50:19.676ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
25812024-11-18T18:50:19.720ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
25822024-11-18T18:50:19.789ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 112441 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:47274 req_id = 9c9395c4-2343-4bec-9403-a3328896ea63 response_code = 204 uri = /switch-ports
25832024-11-18T18:50:19.789ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:58080
25842024-11-18T18:50:19.789ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:37549
25852024-11-18T18:50:19.789ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
25862024-11-18T18:50:19.793ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 4170 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:47274 req_id = 6ed20556-97e5-4e66-86c3-2362ed427b71 response_code = 200 uri = /network-bootstore-config
25872024-11-18T18:50:19.794ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 4353 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:58080 req_id = c779bebe-3fca-4c25-b6af-3c23c14af066 response_code = 200 uri = /vpc-routes
25882024-11-18T18:50:19.820ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
25892024-11-18T18:50:19.820ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 130 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:58080 req_id = fc33ed92-6ddb-408f-b4c9-cfc0b9c1df3d response_code = 204 uri = /eip-gateways
25902024-11-18T18:50:19.831ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:46729
25912024-11-18T18:50:19.832ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 249 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:46729 req_id = b9a9e602-0f03-4999-b19a-45b8a2bccea3 response_code = 200 uri = /v2p
25922024-11-18T18:50:19.840ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 211 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:58080 req_id = 8ed188f1-73c7-4a9b-81db-0c0c8eb68079 response_code = 204 uri = /vpc-routes
25932024-11-18T18:50:19.841ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
25942024-11-18T18:50:19.904ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 114924 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:37549 req_id = c7c85e6b-6788-4776-be8e-0cfc884f968f response_code = 204 uri = /switch-ports
25952024-11-18T18:50:19.904ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 151 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:37549 req_id = cae8016a-4614-4dc3-9e0d-2c54135697b3 response_code = 200 uri = /network-bootstore-config
25962024-11-18T18:50:19.908ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:49044
25972024-11-18T18:50:19.909ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 117 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:49044 req_id = bc74d86a-6a2c-438d-90ab-9ea1445d1bf9 response_code = 200 uri = /v2p
25982024-11-18T18:50:25.881ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
25992024-11-18T18:50:25.883ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
26002024-11-18T18:50:25.883ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
26012024-11-18T18:50:25.935ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
26022024-11-18T18:50:32.628ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:54220
26032024-11-18T18:50:32.628ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 84 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:54220 req_id = b0e31624-0276-44bb-8977-f9227b19ced9 response_code = 200 uri = /baseboard
26042024-11-18T18:50:35.883ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
26052024-11-18T18:50:35.885ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
26062024-11-18T18:50:35.885ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
26072024-11-18T18:50:35.937ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
26082024-11-18T18:50:45.885ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
26092024-11-18T18:50:45.887ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
26102024-11-18T18:50:45.888ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
26112024-11-18T18:50:45.938ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
26122024-11-18T18:50:48.743ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:64548
26132024-11-18T18:50:48.744ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 86 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:64548 req_id = 3f0cf82e-dc33-483a-b301-f968ac622df9 response_code = 200 uri = /vpc-routes
26142024-11-18T18:50:48.813ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
26152024-11-18T18:50:48.813ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 98 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:64548 req_id = 8a7761cd-6d0d-4920-8634-b3de26fc6597 response_code = 204 uri = /eip-gateways
26162024-11-18T18:50:49.000ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:63953
26172024-11-18T18:50:49.000ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 140 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:63953 req_id = b321470c-975f-4504-8f72-0b8fb20744c1 response_code = 200 uri = /v2p
26182024-11-18T18:50:49.013ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 254 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:64548 req_id = 374e789c-6152-41be-ba89-fa03c21c4129 response_code = 204 uri = /vpc-routes
26192024-11-18T18:50:49.086ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:36020
26202024-11-18T18:50:49.088ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
26212024-11-18T18:50:49.088ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
26222024-11-18T18:50:49.088ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/9216f0f9-29c9-4d33-8c71-0f237e1331a0/config/omicron-zones.json file = common/src/ledger.rs:177
26232024-11-18T18:50:49.089ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/c23ca4a6-dcd8-4835-a278-26664c492a4d/config/omicron-zones.json file = common/src/ledger.rs:177
26242024-11-18T18:50:49.105ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
26252024-11-18T18:50:49.105ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_4be98a0c-ab4c-4309-948b-04d776603876
26262024-11-18T18:50:49.110ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:59954
26272024-11-18T18:50:49.111ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
26282024-11-18T18:50:49.141ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
26292024-11-18T18:50:49.141ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_4f35fd7e-936a-4e09-946b-7cb174580897
26302024-11-18T18:50:49.162ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
26312024-11-18T18:50:49.171ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
26322024-11-18T18:50:49.171ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_df4faf1b-f747-4cda-8481-deaa8523a533
26332024-11-18T18:50:49.199ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 0199d4fd-4090-4a7c-83e6-040814eb79bb (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
26342024-11-18T18:50:49.199ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_0199d4fd-4090-4a7c-83e6-040814eb79bb
26352024-11-18T18:50:49.218ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:50216
26362024-11-18T18:50:49.222ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:731 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 6a2febaf-2bbc-4cc0-aab6-80daf994ec33, kind: Service { id: 6767b855-63e1-4a7d-ac38-c4e7eb56747f }, name: Name("external-dns-6767b855-63e1-4a7d-ac38-c4e7eb56747f"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 152, 29])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 72590ac9-2812-4929-9b37-1829b4168efa, kind: Service { id: ceea2287-3571-41ea-8f15-b2aeb4a74fd9 }, name: Name("external-dns-ceea2287-3571-41ea-8f15-b2aeb4a74fd9"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 170, 19])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: ab6c5d8c-f7d0-4cf6-8c3c-5acf8f3959a9, kind: Service { id: d7ddf21e-0e1e-4004-b302-ecb81ba84c09 }, name: Name("nexus-d7ddf21e-0e1e-4004-b302-ecb81ba84c09"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 151, 123])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d0f6d114-ab1d-41fb-a2bb-c154f579e21f, kind: Service { id: 88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d }, name: Name("nexus-88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 37])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f6afceac-9bde-4010-ba9d-6c2eebc86b2d, kind: Service { id: 1bd1422d-700f-41e4-b299-7268d8edd3b1 }, name: Name("nexus-1bd1422d-700f-41e4-b299-7268d8edd3b1"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 0])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
26372024-11-18T18:50:49.222ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
26382024-11-18T18:50:49.222ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
26392024-11-18T18:50:49.222ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
26402024-11-18T18:50:49.222ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte0" rules = []
26412024-11-18T18:50:49.222ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
26422024-11-18T18:50:49.222ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
26432024-11-18T18:50:49.223ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 4783 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:50216 req_id = 67fd2dc4-a724-4163-b976-f652f19997ad response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
26442024-11-18T18:50:49.226ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 5840d858-3782-45e1-a6e0-d60d070ad2c3 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
26452024-11-18T18:50:49.226ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_5840d858-3782-45e1-a6e0-d60d070ad2c3
26462024-11-18T18:50:49.231ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 120431 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:59954 req_id = 315d6723-1e28-4ada-9ed8-fb6857e9c6da response_code = 204 uri = /switch-ports
26472024-11-18T18:50:49.231ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 245 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:59954 req_id = 1d17a2cb-9163-448d-aee6-4beba45dd481 response_code = 200 uri = /network-bootstore-config
26482024-11-18T18:50:49.251ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 131b4d69-dbb0-4397-9284-176e4a3c973d (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
26492024-11-18T18:50:49.251ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_131b4d69-dbb0-4397-9284-176e4a3c973d
26502024-11-18T18:50:49.274ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 453f7245-5f70-4fe8-b37e-054e56d5c1c5 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
26512024-11-18T18:50:49.274ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_453f7245-5f70-4fe8-b37e-054e56d5c1c5
26522024-11-18T18:50:49.299ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: b09b6e80-5341-4ded-a520-3f312403e198 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
26532024-11-18T18:50:49.299ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_b09b6e80-5341-4ded-a520-3f312403e198
26542024-11-18T18:50:49.334ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: f23dc95a-d65a-4537-b65e-161954e059fa (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
26552024-11-18T18:50:49.334ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_f23dc95a-d65a-4537-b65e-161954e059fa
26562024-11-18T18:50:49.359ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 272881 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:36020 req_id = 657daa40-3ba6-41d9-aced-236aa253407c response_code = 200 uri = /inventory
26572024-11-18T18:50:49.542ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:64372
26582024-11-18T18:50:49.543ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 89 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:64372 req_id = e1d039a6-a79c-4447-8fb7-00e39a61ce93 response_code = 200 uri = /vpc-routes
26592024-11-18T18:50:49.678ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
26602024-11-18T18:50:49.678ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 92 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:64372 req_id = db2531d4-d46b-4c82-abfa-fede7610fa73 response_code = 204 uri = /eip-gateways
26612024-11-18T18:50:49.702ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 278 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:64372 req_id = 4b2c9b87-2bec-4905-bb3f-d53876672c93 response_code = 204 uri = /vpc-routes
26622024-11-18T18:50:49.858ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:47070
26632024-11-18T18:50:49.858ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 57 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:47070 req_id = 7a1bd73e-21c5-4479-9120-41cadbe41378 response_code = 200 uri = /vpc-routes
26642024-11-18T18:50:49.874ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:45679
26652024-11-18T18:50:49.874ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
26662024-11-18T18:50:49.874ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
26672024-11-18T18:50:49.874ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/9216f0f9-29c9-4d33-8c71-0f237e1331a0/config/omicron-zones.json file = common/src/ledger.rs:177
26682024-11-18T18:50:49.874ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/c23ca4a6-dcd8-4835-a278-26664c492a4d/config/omicron-zones.json file = common/src/ledger.rs:177
26692024-11-18T18:50:49.888ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
26702024-11-18T18:50:49.888ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_4be98a0c-ab4c-4309-948b-04d776603876
26712024-11-18T18:50:49.918ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
26722024-11-18T18:50:49.918ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_4f35fd7e-936a-4e09-946b-7cb174580897
26732024-11-18T18:50:49.924ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:59654
26742024-11-18T18:50:49.925ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 186 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:59654 req_id = 2510a21f-0198-4329-b55c-a026b793fc2f response_code = 200 uri = /v2p
26752024-11-18T18:50:49.925ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
26762024-11-18T18:50:49.925ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 64 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:47070 req_id = 38885a2d-7a43-4bf0-984b-bbf71daa49f7 response_code = 204 uri = /eip-gateways
26772024-11-18T18:50:49.940ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 923 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:47070 req_id = 1402f1f5-7911-4ee0-b39d-7b8ea473691a response_code = 204 uri = /vpc-routes
26782024-11-18T18:50:49.941ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:54583
26792024-11-18T18:50:49.941ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
26802024-11-18T18:50:49.948ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
26812024-11-18T18:50:49.948ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_df4faf1b-f747-4cda-8481-deaa8523a533
26822024-11-18T18:50:49.982ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 0199d4fd-4090-4a7c-83e6-040814eb79bb (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
26832024-11-18T18:50:49.982ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_0199d4fd-4090-4a7c-83e6-040814eb79bb
26842024-11-18T18:50:49.989ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
26852024-11-18T18:50:50.003ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:35215
26862024-11-18T18:50:50.003ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:65037
26872024-11-18T18:50:50.004ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 307 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:65037 req_id = a5636d24-4590-47e5-8ab3-35a401a1d614 response_code = 200 uri = /v2p
26882024-11-18T18:50:50.012ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 5840d858-3782-45e1-a6e0-d60d070ad2c3 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
26892024-11-18T18:50:50.012ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_5840d858-3782-45e1-a6e0-d60d070ad2c3
26902024-11-18T18:50:50.058ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 131b4d69-dbb0-4397-9284-176e4a3c973d (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
26912024-11-18T18:50:50.058ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_131b4d69-dbb0-4397-9284-176e4a3c973d
26922024-11-18T18:50:50.067ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
26932024-11-18T18:50:50.067ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 126104 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:54583 req_id = 88881208-fc0a-4cf9-ac6d-23a9c5e8cc84 response_code = 204 uri = /switch-ports
26942024-11-18T18:50:50.072ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 724 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:54583 req_id = f2cccca2-58fe-4818-a02a-2d268b8d0b1d response_code = 200 uri = /network-bootstore-config
26952024-11-18T18:50:50.089ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 453f7245-5f70-4fe8-b37e-054e56d5c1c5 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
26962024-11-18T18:50:50.089ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_453f7245-5f70-4fe8-b37e-054e56d5c1c5
26972024-11-18T18:50:50.109ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
26982024-11-18T18:50:50.117ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: b09b6e80-5341-4ded-a520-3f312403e198 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
26992024-11-18T18:50:50.117ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_b09b6e80-5341-4ded-a520-3f312403e198
27002024-11-18T18:50:50.127ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:37032
27012024-11-18T18:50:50.130ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
27022024-11-18T18:50:50.130ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
27032024-11-18T18:50:50.130ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/9216f0f9-29c9-4d33-8c71-0f237e1331a0/config/omicron-zones.json file = common/src/ledger.rs:177
27042024-11-18T18:50:50.138ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:53333
27052024-11-18T18:50:50.139ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:731 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 6a2febaf-2bbc-4cc0-aab6-80daf994ec33, kind: Service { id: 6767b855-63e1-4a7d-ac38-c4e7eb56747f }, name: Name("external-dns-6767b855-63e1-4a7d-ac38-c4e7eb56747f"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 152, 29])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 72590ac9-2812-4929-9b37-1829b4168efa, kind: Service { id: ceea2287-3571-41ea-8f15-b2aeb4a74fd9 }, name: Name("external-dns-ceea2287-3571-41ea-8f15-b2aeb4a74fd9"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 170, 19])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: ab6c5d8c-f7d0-4cf6-8c3c-5acf8f3959a9, kind: Service { id: d7ddf21e-0e1e-4004-b302-ecb81ba84c09 }, name: Name("nexus-d7ddf21e-0e1e-4004-b302-ecb81ba84c09"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 151, 123])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d0f6d114-ab1d-41fb-a2bb-c154f579e21f, kind: Service { id: 88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d }, name: Name("nexus-88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 37])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f6afceac-9bde-4010-ba9d-6c2eebc86b2d, kind: Service { id: 1bd1422d-700f-41e4-b299-7268d8edd3b1 }, name: Name("nexus-1bd1422d-700f-41e4-b299-7268d8edd3b1"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 0])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
27062024-11-18T18:50:50.139ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
27072024-11-18T18:50:50.139ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
27082024-11-18T18:50:50.139ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27092024-11-18T18:50:50.139ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte0" rules = []
27102024-11-18T18:50:50.139ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27112024-11-18T18:50:50.139ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27122024-11-18T18:50:50.139ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 604 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:53333 req_id = 8c61c311-6bbd-430c-b8d8-f2eb3f4e43f0 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
27132024-11-18T18:50:50.145ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/c23ca4a6-dcd8-4835-a278-26664c492a4d/config/omicron-zones.json file = common/src/ledger.rs:177
27142024-11-18T18:50:50.152ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: f23dc95a-d65a-4537-b65e-161954e059fa (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
27152024-11-18T18:50:50.152ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_f23dc95a-d65a-4537-b65e-161954e059fa
27162024-11-18T18:50:50.169ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 4be98a0c-ab4c-4309-948b-04d776603876 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
27172024-11-18T18:50:50.169ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_4be98a0c-ab4c-4309-948b-04d776603876
27182024-11-18T18:50:50.198ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 194827 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:35215 req_id = d7d7b748-ae8b-4387-a574-c29d9b3ba627 response_code = 204 uri = /switch-ports
27192024-11-18T18:50:50.199ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 325244 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:45679 req_id = 0a98e836-1a8c-48f2-a8b4-74166280285f response_code = 200 uri = /inventory
27202024-11-18T18:50:50.199ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 317 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:35215 req_id = a29137f9-7c6d-47ae-bad9-f74fb83808ab response_code = 200 uri = /network-bootstore-config
27212024-11-18T18:50:50.208ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 4f35fd7e-936a-4e09-946b-7cb174580897 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
27222024-11-18T18:50:50.208ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_4f35fd7e-936a-4e09-946b-7cb174580897
27232024-11-18T18:50:50.239ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: df4faf1b-f747-4cda-8481-deaa8523a533 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
27242024-11-18T18:50:50.240ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_df4faf1b-f747-4cda-8481-deaa8523a533
27252024-11-18T18:50:50.257ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:64526
27262024-11-18T18:50:50.258ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:731 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 6a2febaf-2bbc-4cc0-aab6-80daf994ec33, kind: Service { id: 6767b855-63e1-4a7d-ac38-c4e7eb56747f }, name: Name("external-dns-6767b855-63e1-4a7d-ac38-c4e7eb56747f"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 152, 29])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 72590ac9-2812-4929-9b37-1829b4168efa, kind: Service { id: ceea2287-3571-41ea-8f15-b2aeb4a74fd9 }, name: Name("external-dns-ceea2287-3571-41ea-8f15-b2aeb4a74fd9"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 170, 19])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: ab6c5d8c-f7d0-4cf6-8c3c-5acf8f3959a9, kind: Service { id: d7ddf21e-0e1e-4004-b302-ecb81ba84c09 }, name: Name("nexus-d7ddf21e-0e1e-4004-b302-ecb81ba84c09"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 151, 123])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: d0f6d114-ab1d-41fb-a2bb-c154f579e21f, kind: Service { id: 88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d }, name: Name("nexus-88db8d2a-ff0a-4c62-ad5a-f2ac4127dd3d"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 37])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f6afceac-9bde-4010-ba9d-6c2eebc86b2d, kind: Service { id: 1bd1422d-700f-41e4-b299-7268d8edd3b1 }, name: Name("nexus-1bd1422d-700f-41e4-b299-7268d8edd3b1"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 0])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
27272024-11-18T18:50:50.258ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
27282024-11-18T18:50:50.258ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
27292024-11-18T18:50:50.258ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27302024-11-18T18:50:50.258ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte0" rules = []
27312024-11-18T18:50:50.258ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27322024-11-18T18:50:50.258ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27332024-11-18T18:50:50.258ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 506 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:64526 req_id = 8afa9f1d-b013-4745-b366-2ad0806ea9d2 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
27342024-11-18T18:50:50.264ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 0199d4fd-4090-4a7c-83e6-040814eb79bb (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
27352024-11-18T18:50:50.264ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_0199d4fd-4090-4a7c-83e6-040814eb79bb
27362024-11-18T18:50:50.288ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 5840d858-3782-45e1-a6e0-d60d070ad2c3 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
27372024-11-18T18:50:50.288ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_5840d858-3782-45e1-a6e0-d60d070ad2c3
27382024-11-18T18:50:50.311ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 131b4d69-dbb0-4397-9284-176e4a3c973d (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
27392024-11-18T18:50:50.311ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_131b4d69-dbb0-4397-9284-176e4a3c973d
27402024-11-18T18:50:50.335ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 453f7245-5f70-4fe8-b37e-054e56d5c1c5 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
27412024-11-18T18:50:50.335ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_453f7245-5f70-4fe8-b37e-054e56d5c1c5
27422024-11-18T18:50:50.360ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: b09b6e80-5341-4ded-a520-3f312403e198 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
27432024-11-18T18:50:50.360ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_b09b6e80-5341-4ded-a520-3f312403e198
27442024-11-18T18:50:50.384ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: f23dc95a-d65a-4537-b65e-161954e059fa (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
27452024-11-18T18:50:50.384ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_f23dc95a-d65a-4537-b65e-161954e059fa
27462024-11-18T18:50:50.398ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 270472 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:37032 req_id = 1ad43bed-4122-4c2d-848e-0f1c54b55e4c response_code = 200 uri = /inventory
27472024-11-18T18:50:55.886ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
27482024-11-18T18:50:55.888ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
27492024-11-18T18:50:55.889ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
27502024-11-18T18:50:55.940ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
27512024-11-18T18:51:02.661ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:59035
27522024-11-18T18:51:02.661ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 85 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:59035 req_id = c991734c-4593-479f-824a-b72b1f6d672c response_code = 200 uri = /baseboard
27532024-11-18T18:51:05.889ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
27542024-11-18T18:51:05.891ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
27552024-11-18T18:51:05.892ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
27562024-11-18T18:51:05.942ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
27572024-11-18T18:51:15.891ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
27582024-11-18T18:51:15.893ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
27592024-11-18T18:51:15.893ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
27602024-11-18T18:51:15.944ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
27612024-11-18T18:51:18.539ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:54087
27622024-11-18T18:51:18.539ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 67 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:54087 req_id = 03eea0cf-748e-4104-b31b-34de104f3ac2 response_code = 200 uri = /vpc-routes
27632024-11-18T18:51:18.560ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
27642024-11-18T18:51:18.560ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 65 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:54087 req_id = 55099386-5519-4273-9415-7aa798268cc3 response_code = 204 uri = /eip-gateways
27652024-11-18T18:51:18.568ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 155 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:54087 req_id = f2de853f-d036-4b3e-911a-731b76d6e776 response_code = 204 uri = /vpc-routes
27662024-11-18T18:51:18.621ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:56538
27672024-11-18T18:51:18.621ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
27682024-11-18T18:51:18.661ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
27692024-11-18T18:51:18.724ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 103171 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:56538 req_id = a150d69c-65c8-496b-8975-7e542bab7941 response_code = 204 uri = /switch-ports
27702024-11-18T18:51:18.725ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:48548
27712024-11-18T18:51:18.725ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
27722024-11-18T18:51:18.725ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 206 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:48548 req_id = 95525c11-1a36-46d4-937c-855dda18ca02 response_code = 200 uri = /v2p
27732024-11-18T18:51:18.725ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 247 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:56538 req_id = 54f0732c-9e2d-4d21-9b83-f1924af7359b response_code = 200 uri = /network-bootstore-config
27742024-11-18T18:51:19.422ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:63515
27752024-11-18T18:51:19.422ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 53 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:63515 req_id = 50d0a618-5eb2-4ece-be30-e81e1dd371d5 response_code = 200 uri = /vpc-routes
27762024-11-18T18:51:19.549ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
27772024-11-18T18:51:19.549ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 86 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:63515 req_id = 5c0fa0d6-6328-487d-b770-4516fd246b5c response_code = 204 uri = /eip-gateways
27782024-11-18T18:51:19.563ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 227 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:63515 req_id = 9d9eb5f8-e452-420d-a23d-25a8c119baef response_code = 204 uri = /vpc-routes
27792024-11-18T18:51:19.607ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:40904
27802024-11-18T18:51:19.607ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
27812024-11-18T18:51:19.656ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
27822024-11-18T18:51:19.725ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 117779 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:40904 req_id = b07d1409-613d-4af4-9ee0-2627b9528960 response_code = 204 uri = /switch-ports
27832024-11-18T18:51:19.725ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:44237
27842024-11-18T18:51:19.725ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 58 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:44237 req_id = bf8693a3-e8cd-4b4c-8e56-675c66538662 response_code = 200 uri = /vpc-routes
27852024-11-18T18:51:19.725ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 97 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:40904 req_id = 4e924163-09ef-49ad-bd57-7d1b6e1a5b3d response_code = 200 uri = /network-bootstore-config
27862024-11-18T18:51:19.726ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:63283
27872024-11-18T18:51:19.727ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
27882024-11-18T18:51:19.770ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
27892024-11-18T18:51:19.827ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 100066 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:63283 req_id = 07fd87c9-28b3-4b82-a3c0-9e3dffc7cb44 response_code = 204 uri = /switch-ports
27902024-11-18T18:51:19.827ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:49049
27912024-11-18T18:51:19.827ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
27922024-11-18T18:51:19.827ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 242 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:44237 req_id = 01c2fe39-ae61-4a23-ab38-5981f97c74aa response_code = 204 uri = /eip-gateways
27932024-11-18T18:51:19.827ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 219 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:63283 req_id = 1a3752c4-046d-4dd4-b161-3f3f26bcb5b6 response_code = 200 uri = /network-bootstore-config
27942024-11-18T18:51:19.827ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 218 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:49049 req_id = a0dd5ebd-12ce-431d-987a-5f46e86c62be response_code = 200 uri = /v2p
27952024-11-18T18:51:19.837ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:40658
27962024-11-18T18:51:19.837ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 172 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:40658 req_id = 3896e1b7-55d3-4a6f-8be3-136a367296e6 response_code = 200 uri = /v2p
27972024-11-18T18:51:19.837ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 202 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:44237 req_id = 4f082c1b-c347-40bc-b666-606f999dcde1 response_code = 204 uri = /vpc-routes
27982024-11-18T18:51:25.893ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
27992024-11-18T18:51:25.895ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
28002024-11-18T18:51:25.895ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
28012024-11-18T18:51:25.946ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
28022024-11-18T18:51:32.695ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:60061
28032024-11-18T18:51:32.695ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 90 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:60061 req_id = d98e447e-5e16-4d87-9065-195f470c72da response_code = 200 uri = /baseboard
28042024-11-18T18:51:35.895ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
28052024-11-18T18:51:35.897ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
28062024-11-18T18:51:35.897ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
28072024-11-18T18:51:35.948ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
28082024-11-18T18:51:45.896ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
28092024-11-18T18:51:45.899ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
28102024-11-18T18:51:45.900ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
28112024-11-18T18:51:45.949ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
28122024-11-18T18:51:48.554ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:35265
28132024-11-18T18:51:48.554ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 51 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:35265 req_id = d6757f45-b42d-4f03-97c7-b87d25c76c9e response_code = 200 uri = /vpc-routes
28142024-11-18T18:51:48.650ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
28152024-11-18T18:51:48.650ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 117 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:35265 req_id = 608ff61d-3b40-40a1-a8b8-cd4b8b239014 response_code = 204 uri = /eip-gateways
28162024-11-18T18:51:48.663ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 249 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:35265 req_id = d8df63fd-c6c9-4be1-8829-8d4308c1e30e response_code = 204 uri = /vpc-routes
28172024-11-18T18:51:48.993ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:52642
28182024-11-18T18:51:48.993ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 134 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:52642 req_id = f5ba2f1d-367c-4cad-a52d-362feff79e86 response_code = 200 uri = /v2p
28192024-11-18T18:51:49.067ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:37808
28202024-11-18T18:51:49.067ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
28212024-11-18T18:51:49.109ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
28222024-11-18T18:51:49.171ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 103657 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:37808 req_id = 4b0d1aea-0337-4f40-915b-7c4d5fbbaa8f response_code = 204 uri = /switch-ports
28232024-11-18T18:51:49.173ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 98 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:37808 req_id = 7bc80af1-b5ba-42ea-b2a4-6435ec0fcdfe response_code = 200 uri = /network-bootstore-config
28242024-11-18T18:51:49.603ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:64482
28252024-11-18T18:51:49.603ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 58 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:64482 req_id = 91901ea8-7dee-48b6-b6e7-8dbef360b8be response_code = 200 uri = /vpc-routes
28262024-11-18T18:51:49.681ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
28272024-11-18T18:51:49.681ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 114 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:64482 req_id = 9f45b9ad-e441-4114-acbb-468f5560ff16 response_code = 204 uri = /eip-gateways
28282024-11-18T18:51:49.691ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 197 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:64482 req_id = 6f2079ab-e9ac-412b-9158-4345a00068a0 response_code = 204 uri = /vpc-routes
28292024-11-18T18:51:49.839ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:53118
28302024-11-18T18:51:49.839ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 33 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:53118 req_id = 82bb20ed-04d7-46f8-b79f-83d7f704f931 response_code = 200 uri = /vpc-routes
28312024-11-18T18:51:49.911ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
28322024-11-18T18:51:49.911ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 65 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:53118 req_id = 579eb8a9-bce1-4376-a4e9-78c7a6305cf0 response_code = 204 uri = /eip-gateways
28332024-11-18T18:51:49.924ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 162 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:53118 req_id = 00754bac-68f8-4607-8ac3-6bfd90396957 response_code = 204 uri = /vpc-routes
28342024-11-18T18:51:49.937ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:57949
28352024-11-18T18:51:49.937ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 121 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:57949 req_id = 8ee09489-1be0-480d-b4e6-bf10ad0190e8 response_code = 200 uri = /v2p
28362024-11-18T18:51:49.952ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:38435
28372024-11-18T18:51:49.952ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
28382024-11-18T18:51:50.000ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
28392024-11-18T18:51:50.078ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 126381 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:38435 req_id = 873a8da2-6d41-454e-88f4-4f08433a8c9d response_code = 204 uri = /switch-ports
28402024-11-18T18:51:50.078ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:56569
28412024-11-18T18:51:50.079ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
28422024-11-18T18:51:50.124ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
28432024-11-18T18:51:50.185ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 106006 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:56569 req_id = 15af2247-4afe-4bb8-b515-4312ce6d6585 response_code = 204 uri = /switch-ports
28442024-11-18T18:51:50.185ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:38916
28452024-11-18T18:51:50.185ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 293 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:38435 req_id = 33cad247-a3a2-428d-9cad-f878f44ae6e1 response_code = 200 uri = /network-bootstore-config
28462024-11-18T18:51:50.185ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 247 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:56569 req_id = ee3227de-fcb8-4b7e-b6ad-b45ed6404404 response_code = 200 uri = /network-bootstore-config
28472024-11-18T18:51:50.185ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 231 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:38916 req_id = e484bd38-c717-4baa-a1c9-43eca0924e40 response_code = 200 uri = /v2p
28482024-11-18T18:51:55.898ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
28492024-11-18T18:51:55.901ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
28502024-11-18T18:51:55.901ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
28512024-11-18T18:51:55.951ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
28522024-11-18T18:52:01.105ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:37091
28532024-11-18T18:52:01.106ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 183 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:37091 req_id = cb164c90-b3a8-4888-8dcc-d1ff771f25c1 response_code = 200 uri = /v2p
28542024-11-18T18:52:01.106ZINFOSledAgent (PortManager): Mapping virtual NIC to physical host file = illumos-utils/src/opte/port_manager.rs:838 mapping = VirtualNetworkInterfaceHost { virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), physical_host_ip: fd00:1122:3344:101::1, vni: Vni(6772514) }
28552024-11-18T18:52:01.107ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 628 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:37091 req_id = 480fabf8-ca11-444c-8185-45b04ea114b1 response_code = 204 uri = /v2p
28562024-11-18T18:52:01.198ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:55503
28572024-11-18T18:52:01.206ZINFOSledAgent (InstanceManager): ensuring instance is registered file = sled-agent/src/instance_manager.rs:555 hardware = InstanceHardware { properties: InstanceProperties { ncpus: InstanceCpuCount(2), memory: ByteCount(1073741824), hostname: Hostname("localshark") }, nics: [NetworkInterface { id: 115fdfc9-52ac-4bdc-bbd6-b0b3711a1cc2, kind: Instance { id: 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(6772514), primary: true, slot: 0, transit_ips: [] }], source_nat: SourceNatConfig { ip: 10.151.2.175, first_port: 0, last_port: 16383 }, ephemeral_ip: Some(10.151.2.176), floating_ips: [], firewall_rules: [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 115fdfc9-52ac-4bdc-bbd6-b0b3711a1cc2, kind: Instance { id: 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(6772514), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 115fdfc9-52ac-4bdc-bbd6-b0b3711a1cc2, kind: Instance { id: 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(6772514), primary: true, slot: 0, transit_ips: [] }], filter_hosts: Some([Vpc(Vni(6772514))]), filter_ports: None, filter_protocols: None, action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 115fdfc9-52ac-4bdc-bbd6-b0b3711a1cc2, kind: Instance { id: 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(6772514), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(22), last: L4Port(22) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }], dhcp_config: DhcpConfig { dns_servers: [1.1.1.1, 9.9.9.9], host_domain: None, search_domains: [] }, disks: [DiskRequest { device: "nvme", name: "disk-5c5cdf34cc7", read_only: false, slot: Slot(0), volume_construction_request: Volume { block_size: 512, id: 959a5fab-0846-40c1-b624-0bd77544efb3, read_only_parent: Some(Volume { block_size: 512, id: 13796b3e-ce0f-4f72-8fea-492e2b768d00, read_only_parent: None, sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 2, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: ad8ba3ca-d508-45fc-b35b-dc1328fdcb42, key: Some("kgdC2ifZdoNDMzRFet6rZhgDWTT5E8fpHBoN9Jp9eOc="), key_pem: None, lossy: false, read_only: true, root_cert_pem: None, target: ["[fd00:1122:3344:101::17]:19001", "[fd00:1122:3344:101::14]:19001", "[fd00:1122:3344:101::1a]:19001"] } }] }), sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 1, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 959a5fab-0846-40c1-b624-0bd77544efb3, key: Some("Dh5JtIP577fbYFbhqmGT5PWCMf3Bg7aAK6Nf5QrxdFk="), key_pem: None, lossy: false, read_only: false, root_cert_pem: None, target: ["[fd00:1122:3344:101::14]:19002", "[fd00:1122:3344:101::15]:19000", "[fd00:1122:3344:101::16]:19000"] } }] } }], boot_settings: Some(BootSettings { order: [BootOrderEntry { name: "disk-5c5cdf34cc7" }] }), cloud_init_bytes: Some(..) } instance_id = 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 metadata = InstanceMetadata { silo_id: 14231163-1e25-413b-b811-7abbeb5b91a3, project_id: 5880f294-59dd-4560-a265-cb5419900e36 } migration_id = None propolis_addr = [fd00:1122:3344:101::1:0]:12400 propolis_id = a69389c5-8d0a-47da-b47d-3c8aacfaa217 vmm_runtime = VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2024-11-18T18:52:01.196130091Z }
28582024-11-18T18:52:01.206ZINFOSledAgent (InstanceManager): registering new instance file = sled-agent/src/instance_manager.rs:588 instance_id = 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 migration_id = None propolis_id = a69389c5-8d0a-47da-b47d-3c8aacfaa217
28592024-11-18T18:52:01.206ZINFOSledAgent (InstanceManager): initializing new Instance file = sled-agent/src/instance.rs:1281 instance_id = 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 migration_id = None propolis_id = a69389c5-8d0a-47da-b47d-3c8aacfaa217 state = InstanceInitialState { hardware: InstanceHardware { properties: InstanceProperties { ncpus: InstanceCpuCount(2), memory: ByteCount(1073741824), hostname: Hostname("localshark") }, nics: [NetworkInterface { id: 115fdfc9-52ac-4bdc-bbd6-b0b3711a1cc2, kind: Instance { id: 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(6772514), primary: true, slot: 0, transit_ips: [] }], source_nat: SourceNatConfig { ip: 10.151.2.175, first_port: 0, last_port: 16383 }, ephemeral_ip: Some(10.151.2.176), floating_ips: [], firewall_rules: [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 115fdfc9-52ac-4bdc-bbd6-b0b3711a1cc2, kind: Instance { id: 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(6772514), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 115fdfc9-52ac-4bdc-bbd6-b0b3711a1cc2, kind: Instance { id: 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(6772514), primary: true, slot: 0, transit_ips: [] }], filter_hosts: Some([Vpc(Vni(6772514))]), filter_ports: None, filter_protocols: None, action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 115fdfc9-52ac-4bdc-bbd6-b0b3711a1cc2, kind: Instance { id: 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(6772514), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(22), last: L4Port(22) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }], dhcp_config: DhcpConfig { dns_servers: [1.1.1.1, 9.9.9.9], host_domain: None, search_domains: [] }, disks: [DiskRequest { device: "nvme", name: "disk-5c5cdf34cc7", read_only: false, slot: Slot(0), volume_construction_request: Volume { block_size: 512, id: 959a5fab-0846-40c1-b624-0bd77544efb3, read_only_parent: Some(Volume { block_size: 512, id: 13796b3e-ce0f-4f72-8fea-492e2b768d00, read_only_parent: None, sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 2, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: ad8ba3ca-d508-45fc-b35b-dc1328fdcb42, key: Some("kgdC2ifZdoNDMzRFet6rZhgDWTT5E8fpHBoN9Jp9eOc="), key_pem: None, lossy: false, read_only: true, root_cert_pem: None, target: ["[fd00:1122:3344:101::17]:19001", "[fd00:1122:3344:101::14]:19001", "[fd00:1122:3344:101::1a]:19001"] } }] }), sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 1, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 959a5fab-0846-40c1-b624-0bd77544efb3, key: Some("Dh5JtIP577fbYFbhqmGT5PWCMf3Bg7aAK6Nf5QrxdFk="), key_pem: None, lossy: false, read_only: false, root_cert_pem: None, target: ["[fd00:1122:3344:101::14]:19002", "[fd00:1122:3344:101::15]:19000", "[fd00:1122:3344:101::16]:19000"] } }] } }], boot_settings: Some(BootSettings { order: [BootOrderEntry { name: "disk-5c5cdf34cc7" }] }), cloud_init_bytes: Some(..) }, vmm_runtime: VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2024-11-18T18:52:01.196130091Z }, propolis_addr: [fd00:1122:3344:101::1:0]:12400, migration_id: None }
28602024-11-18T18:52:01.208ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 9730 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:55503 req_id = 97b8654f-02ba-48dd-a325-a9558046416d response_code = 200 uri = /vmms/a69389c5-8d0a-47da-b47d-3c8aacfaa217
28612024-11-18T18:52:01.283ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:51532
28622024-11-18T18:52:01.283ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 164 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:51532 req_id = be000b6b-d795-4138-b15d-2feddfd8a982 response_code = 200 uri = /vmms/a69389c5-8d0a-47da-b47d-3c8aacfaa217/state
28632024-11-18T18:52:01.284ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:430 port = Port { inner: PortInner(PortData { name: "opte6", ip: 172.30.0.5, mac: MacAddr6([168, 64, 37, 240, 0, 0]), slot: 0, vni: Vni { inner: 6772514 }, subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.0.1 } }) }
28642024-11-18T18:52:01.284ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
28652024-11-18T18:52:01.331ZINFOSledAgent (InstanceManager): Configuring new Omicron zone: oxz_propolis-server_a69389c5-8d0a-47da-b47d-3c8aacfaa217 file = illumos-utils/src/zone.rs:325 instance_id = 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 propolis_id = a69389c5-8d0a-47da-b47d-3c8aacfaa217
28662024-11-18T18:52:01.357ZINFOSledAgent (InstanceManager): Installing Omicron zone: oxz_propolis-server_a69389c5-8d0a-47da-b47d-3c8aacfaa217 file = illumos-utils/src/zone.rs:363 instance_id = 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 propolis_id = a69389c5-8d0a-47da-b47d-3c8aacfaa217
28672024-11-18T18:52:01.496ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:58078
28682024-11-18T18:52:01.497ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 94 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:58078 req_id = 94948a2f-2308-4466-98ee-6705e0ae77c0 response_code = 200 uri = /vpc-routes
28692024-11-18T18:52:01.520ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {115fdfc9-52ac-4bdc-bbd6-b0b3711a1cc2: {10.151.2.175: {92fb4b71-bd15-4124-8f58-24a55855b40c}, 10.151.2.176: {92fb4b71-bd15-4124-8f58-24a55855b40c}}} }
28702024-11-18T18:52:02.706ZINFOSledAgent (InstanceManager): Profile for oxz_propolis-server_a69389c5-8d0a-47da-b47d-3c8aacfaa217: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="system/illumos/propolis-server"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlInstance0'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::1:0'/> <propval type="astring" name="listen_port" value='12400'/> <propval type="astring" name="metric_addr" value='dns'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34 instance_id = 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 propolis_id = a69389c5-8d0a-47da-b47d-3c8aacfaa217
28712024-11-18T18:52:02.706ZINFOSledAgent (InstanceManager): Booting oxz_propolis-server_a69389c5-8d0a-47da-b47d-3c8aacfaa217 zone file = illumos-utils/src/running_zone.rs:488 instance_id = 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 propolis_id = a69389c5-8d0a-47da-b47d-3c8aacfaa217 zone = oxz_propolis-server_a69389c5-8d0a-47da-b47d-3c8aacfaa217
28722024-11-18T18:52:02.745ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:56291
28732024-11-18T18:52:02.745ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 121 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:56291 req_id = da8cba14-1e71-4c14-8afa-acf3cf682410 response_code = 200 uri = /baseboard
28742024-11-18T18:52:02.961ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_a69389c5-8d0a-47da-b47d-3c8aacfaa217") failed: Property not found. retry in 57.850829ms file = illumos-utils/src/svc.rs:36 instance_id = 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 propolis_id = a69389c5-8d0a-47da-b47d-3c8aacfaa217 zone = oxz_propolis-server_a69389c5-8d0a-47da-b47d-3c8aacfaa217
28752024-11-18T18:52:03.053ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_a69389c5-8d0a-47da-b47d-3c8aacfaa217") failed: Property not found. retry in 86.078732ms file = illumos-utils/src/svc.rs:36 instance_id = 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 propolis_id = a69389c5-8d0a-47da-b47d-3c8aacfaa217 zone = oxz_propolis-server_a69389c5-8d0a-47da-b47d-3c8aacfaa217
28762024-11-18T18:52:03.157ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_a69389c5-8d0a-47da-b47d-3c8aacfaa217") failed: Property not found. retry in 200.501797ms file = illumos-utils/src/svc.rs:36 instance_id = 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 propolis_id = a69389c5-8d0a-47da-b47d-3c8aacfaa217 zone = oxz_propolis-server_a69389c5-8d0a-47da-b47d-3c8aacfaa217
28772024-11-18T18:52:03.376ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_a69389c5-8d0a-47da-b47d-3c8aacfaa217") failed: Property not found. retry in 570.002592ms file = illumos-utils/src/svc.rs:36 instance_id = 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 propolis_id = a69389c5-8d0a-47da-b47d-3c8aacfaa217 zone = oxz_propolis-server_a69389c5-8d0a-47da-b47d-3c8aacfaa217
28782024-11-18T18:52:03.966ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_a69389c5-8d0a-47da-b47d-3c8aacfaa217") failed: Property not found. retry in 574.367343ms file = illumos-utils/src/svc.rs:36 instance_id = 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 propolis_id = a69389c5-8d0a-47da-b47d-3c8aacfaa217 zone = oxz_propolis-server_a69389c5-8d0a-47da-b47d-3c8aacfaa217
28792024-11-18T18:52:04.556ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_a69389c5-8d0a-47da-b47d-3c8aacfaa217") failed: Property not found. retry in 1.176177356s file = illumos-utils/src/svc.rs:36 instance_id = 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 propolis_id = a69389c5-8d0a-47da-b47d-3c8aacfaa217 zone = oxz_propolis-server_a69389c5-8d0a-47da-b47d-3c8aacfaa217
28802024-11-18T18:52:05.744ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_a69389c5-8d0a-47da-b47d-3c8aacfaa217") failed: Property not found. retry in 894.160289ms file = illumos-utils/src/svc.rs:36 instance_id = 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 propolis_id = a69389c5-8d0a-47da-b47d-3c8aacfaa217 zone = oxz_propolis-server_a69389c5-8d0a-47da-b47d-3c8aacfaa217
28812024-11-18T18:52:05.900ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
28822024-11-18T18:52:05.903ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
28832024-11-18T18:52:05.904ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
28842024-11-18T18:52:05.953ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
28852024-11-18T18:52:06.651ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_a69389c5-8d0a-47da-b47d-3c8aacfaa217") failed: Property not found. retry in 937.898322ms file = illumos-utils/src/svc.rs:36 instance_id = 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 propolis_id = a69389c5-8d0a-47da-b47d-3c8aacfaa217 zone = oxz_propolis-server_a69389c5-8d0a-47da-b47d-3c8aacfaa217
28862024-11-18T18:52:07.625ZINFOSledAgent (InstanceManager): Started propolis in zone: oxz_propolis-server_a69389c5-8d0a-47da-b47d-3c8aacfaa217 file = sled-agent/src/instance.rs:1787 instance_id = 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 propolis_id = a69389c5-8d0a-47da-b47d-3c8aacfaa217
28872024-11-18T18:52:07.637ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_a69389c5-8d0a-47da-b47d-3c8aacfaa217") failed: Property not found. retry in 48.214685ms file = illumos-utils/src/svc.rs:36 instance_id = 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 propolis_id = a69389c5-8d0a-47da-b47d-3c8aacfaa217
28882024-11-18T18:52:07.699ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_a69389c5-8d0a-47da-b47d-3c8aacfaa217") failed: Property not found. retry in 119.958969ms file = illumos-utils/src/svc.rs:36 instance_id = 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 propolis_id = a69389c5-8d0a-47da-b47d-3c8aacfaa217
28892024-11-18T18:52:07.834ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_a69389c5-8d0a-47da-b47d-3c8aacfaa217") failed: Property not found. retry in 278.810051ms file = illumos-utils/src/svc.rs:36 instance_id = 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 propolis_id = a69389c5-8d0a-47da-b47d-3c8aacfaa217
28902024-11-18T18:52:08.127ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_a69389c5-8d0a-47da-b47d-3c8aacfaa217") failed: Property not found. retry in 534.883921ms file = illumos-utils/src/svc.rs:36 instance_id = 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 propolis_id = a69389c5-8d0a-47da-b47d-3c8aacfaa217
28912024-11-18T18:52:08.675ZINFOSledAgent (InstanceManager): Propolis SMF service is online file = sled-agent/src/instance.rs:1796 instance_id = 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 propolis_id = a69389c5-8d0a-47da-b47d-3c8aacfaa217
28922024-11-18T18:52:08.713ZINFOSledAgent (InstanceManager): Propolis HTTP server online file = sled-agent/src/instance.rs:1820 instance_id = 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 propolis_id = a69389c5-8d0a-47da-b47d-3c8aacfaa217
28932024-11-18T18:52:08.853ZINFOSledAgent (InstanceManager): result of instance_ensure call is Ok(InstanceEnsureResponse { migrate: None }) file = sled-agent/src/instance.rs:932 instance_id = 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 propolis_id = a69389c5-8d0a-47da-b47d-3c8aacfaa217
28942024-11-18T18:52:08.854ZINFOSledAgent: IGW mapping changed; external IPs refreshed file = sled-agent/src/sled_agent.rs:1231 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
28952024-11-18T18:52:08.854ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7333588 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:58078 req_id = de704d66-2404-4dbb-8aba-6fc74cd31258 response_code = 204 uri = /eip-gateways
28962024-11-18T18:52:08.855ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Creating), migration_in: None, migration_out: None, time: 2024-11-18T18:52:08.855928381Z } file = sled-agent/src/instance.rs:780 instance_id = 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 propolis_id = a69389c5-8d0a-47da-b47d-3c8aacfaa217
28972024-11-18T18:52:08.855ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:802 instance_id = 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 new_vmm_state = VmmRuntimeState { state: Starting, gen: Generation(3), time_updated: 2024-11-18T18:52:08.855928381Z } propolis_id = a69389c5-8d0a-47da-b47d-3c8aacfaa217
28982024-11-18T18:52:08.856ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 propolis_id = a69389c5-8d0a-47da-b47d-3c8aacfaa217 state = SledVmmState { vmm_state: VmmRuntimeState { state: Starting, gen: Generation(3), time_updated: 2024-11-18T18:52:08.855928381Z }, migration_in: None, migration_out: None }
28992024-11-18T18:52:08.892ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Running), migration_in: None, migration_out: None, time: 2024-11-18T18:52:08.892059779Z } file = sled-agent/src/instance.rs:780 instance_id = 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 propolis_id = a69389c5-8d0a-47da-b47d-3c8aacfaa217
29002024-11-18T18:52:08.892ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:802 instance_id = 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 new_vmm_state = VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2024-11-18T18:52:08.892059779Z } propolis_id = a69389c5-8d0a-47da-b47d-3c8aacfaa217
29012024-11-18T18:52:08.892ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 propolis_id = a69389c5-8d0a-47da-b47d-3c8aacfaa217 state = SledVmmState { vmm_state: VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2024-11-18T18:52:08.892059779Z }, migration_in: None, migration_out: None }
29022024-11-18T18:52:08.949ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 280 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:58078 req_id = 01a92e92-49fd-46a2-b4cb-1064297babd6 response_code = 204 uri = /vpc-routes
29032024-11-18T18:52:15.901ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
29042024-11-18T18:52:15.904ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
29052024-11-18T18:52:15.905ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
29062024-11-18T18:52:15.955ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
29072024-11-18T18:52:18.558ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:59185
29082024-11-18T18:52:18.559ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 71 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:59185 req_id = cc510d58-c443-466d-b03e-65394f63a3f6 response_code = 200 uri = /vpc-routes
29092024-11-18T18:52:18.572ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:61885
29102024-11-18T18:52:18.579ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 169 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:61885 req_id = 447a7227-dd59-4a8a-a980-027022764cae response_code = 200 uri = /vmms/a69389c5-8d0a-47da-b47d-3c8aacfaa217/state
29112024-11-18T18:52:18.640ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {115fdfc9-52ac-4bdc-bbd6-b0b3711a1cc2: {10.151.2.175: {92fb4b71-bd15-4124-8f58-24a55855b40c}, 10.151.2.176: {92fb4b71-bd15-4124-8f58-24a55855b40c}}} }
29122024-11-18T18:52:18.640ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 147 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:59185 req_id = c820c0f1-f6f8-4ac7-a5bc-7524386da0eb response_code = 204 uri = /eip-gateways
29132024-11-18T18:52:18.662ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 282 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:59185 req_id = c8eef260-6ae3-4a9e-8443-60a033906044 response_code = 204 uri = /vpc-routes
29142024-11-18T18:52:18.759ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:65450
29152024-11-18T18:52:18.759ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
29162024-11-18T18:52:18.805ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
29172024-11-18T18:52:18.878ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 119096 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:65450 req_id = 8050d718-efd0-443d-8448-b27a4326113f response_code = 204 uri = /switch-ports
29182024-11-18T18:52:18.879ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 113 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:65450 req_id = 04703aeb-5468-4307-a239-68052b7a7bc1 response_code = 200 uri = /network-bootstore-config
29192024-11-18T18:52:18.936ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:53209
29202024-11-18T18:52:18.937ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 511 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:53209 req_id = de654d97-f5ef-4a5b-a66f-0651a971c3c3 response_code = 200 uri = /vmms/a69389c5-8d0a-47da-b47d-3c8aacfaa217/state
29212024-11-18T18:52:18.937ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Stopping), migration_in: None, migration_out: None, time: 2024-11-18T18:52:18.937637143Z } file = sled-agent/src/instance.rs:780 instance_id = 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 propolis_id = a69389c5-8d0a-47da-b47d-3c8aacfaa217
29222024-11-18T18:52:18.937ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:802 instance_id = 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 new_vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2024-11-18T18:52:18.937637143Z } propolis_id = a69389c5-8d0a-47da-b47d-3c8aacfaa217
29232024-11-18T18:52:18.937ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 propolis_id = a69389c5-8d0a-47da-b47d-3c8aacfaa217 state = SledVmmState { vmm_state: VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2024-11-18T18:52:18.937637143Z }, migration_in: None, migration_out: None }
29242024-11-18T18:52:18.962ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:37403
29252024-11-18T18:52:18.963ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 155 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:37403 req_id = 213eba5a-6dbe-45ad-b93e-5379c55052c3 response_code = 200 uri = /v2p
29262024-11-18T18:52:19.062ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Destroyed), migration_in: None, migration_out: None, time: 2024-11-18T18:52:19.062726613Z } file = sled-agent/src/instance.rs:780 instance_id = 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 propolis_id = a69389c5-8d0a-47da-b47d-3c8aacfaa217
29272024-11-18T18:52:19.062ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:802 instance_id = 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 new_vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(7), time_updated: 2024-11-18T18:52:19.062726613Z } propolis_id = a69389c5-8d0a-47da-b47d-3c8aacfaa217
29282024-11-18T18:52:19.062ZINFOSledAgent (InstanceManager): terminating VMM that has exited file = sled-agent/src/instance.rs:816 instance_id = 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 propolis_id = a69389c5-8d0a-47da-b47d-3c8aacfaa217
29292024-11-18T18:52:19.062ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
29302024-11-18T18:52:19.063ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
29312024-11-18T18:52:19.063ZINFOSledAgent (ZoneBundler): creating zone bundle context = ZoneBundleContext { storage_dirs: ["/pool/int/9216f0f9-29c9-4d33-8c71-0f237e1331a0/debug/bundle/zone", "/pool/int/c23ca4a6-dcd8-4835-a278-26664c492a4d/debug/bundle/zone"], cause: TerminatedInstance, extra_log_dirs: ["/pool/ext/4be98a0c-ab4c-4309-948b-04d776603876/crypt/debug", "/pool/ext/4f35fd7e-936a-4e09-946b-7cb174580897/crypt/debug", "/pool/ext/df4faf1b-f747-4cda-8481-deaa8523a533/crypt/debug", "/pool/ext/0199d4fd-4090-4a7c-83e6-040814eb79bb/crypt/debug", "/pool/ext/5840d858-3782-45e1-a6e0-d60d070ad2c3/crypt/debug", "/pool/ext/131b4d69-dbb0-4397-9284-176e4a3c973d/crypt/debug", "/pool/ext/453f7245-5f70-4fe8-b37e-054e56d5c1c5/crypt/debug", "/pool/ext/b09b6e80-5341-4ded-a520-3f312403e198/crypt/debug", "/pool/ext/f23dc95a-d65a-4537-b65e-161954e059fa/crypt/debug"] } file = sled-agent/src/zone_bundle.rs:339 zone_name = oxz_propolis-server_a69389c5-8d0a-47da-b47d-3c8aacfaa217
29322024-11-18T18:52:19.063ZWARNSledAgent (kstat-sampler-worker): target removed with queued samples file = oximeter/instruments/src/kstat/sampler.rs:602 id = 17393216059368071057 n_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
29332024-11-18T18:52:19.063ZWARNSledAgent (kstat-sampler-worker): target removed with queued samples file = oximeter/instruments/src/kstat/sampler.rs:602 id = 15676142357430512767 n_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
29342024-11-18T18:52:19.552ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51571
29352024-11-18T18:52:19.562ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:46538
29362024-11-18T18:52:19.563ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 76 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:46538 req_id = 0504ca36-da2e-4881-813d-c36b7ac97aa8 response_code = 200 uri = /vpc-routes
29372024-11-18T18:52:19.629ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {115fdfc9-52ac-4bdc-bbd6-b0b3711a1cc2: {10.151.2.176: {92fb4b71-bd15-4124-8f58-24a55855b40c}, 10.151.2.175: {92fb4b71-bd15-4124-8f58-24a55855b40c}}} }
29382024-11-18T18:52:19.629ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 196 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:46538 req_id = a6dce5be-0003-452e-a5ea-4fdb19f76e89 response_code = 204 uri = /eip-gateways
29392024-11-18T18:52:19.633ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:54541
29402024-11-18T18:52:19.649ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 268 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:46538 req_id = 1c5f8977-7ae1-4b81-a109-56ff00800df7 response_code = 204 uri = /vpc-routes
29412024-11-18T18:52:19.664ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:36733
29422024-11-18T18:52:19.664ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
29432024-11-18T18:52:19.711ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
29442024-11-18T18:52:19.780ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:39887
29452024-11-18T18:52:19.780ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:44482
29462024-11-18T18:52:19.786ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 121264 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:36733 req_id = f45b0e95-f90d-4c1f-bc7b-f14ea037b029 response_code = 204 uri = /switch-ports
29472024-11-18T18:52:19.786ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4120
29482024-11-18T18:52:19.786ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 625 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:39887 req_id = 7e0ec255-3a7e-43fd-ba4d-cccc39c3b56e response_code = 200 uri = /vpc-routes
29492024-11-18T18:52:19.793ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 789 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:36733 req_id = da35cb5d-a2ef-4c45-8595-a0db52dcd4d5 response_code = 200 uri = /network-bootstore-config
29502024-11-18T18:52:19.809ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:57636
29512024-11-18T18:52:19.809ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 280 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:57636 req_id = 22d59d2f-1b71-45ee-90c7-bbdb334e5e45 response_code = 200 uri = /v2p
29522024-11-18T18:52:19.836ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4144
29532024-11-18T18:52:19.846ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {115fdfc9-52ac-4bdc-bbd6-b0b3711a1cc2: {10.151.2.176: {92fb4b71-bd15-4124-8f58-24a55855b40c}, 10.151.2.175: {92fb4b71-bd15-4124-8f58-24a55855b40c}}} }
29542024-11-18T18:52:19.846ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 190 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:39887 req_id = 498fe10d-4b1c-4dc0-88cf-5bb902bdab3f response_code = 204 uri = /eip-gateways
29552024-11-18T18:52:19.856ZINFOSledAgent (ZoneBundler): finished zone bundle file = sled-agent/src/zone_bundle.rs:1142 metadata = ZoneBundleMetadata { id: ZoneBundleId { zone_name: "oxz_propolis-server_a69389c5-8d0a-47da-b47d-3c8aacfaa217", bundle_id: af65447d-48bd-4066-b565-4185fc89a158 }, time_created: 2024-11-18T18:52:19.063514788Z, version: 0, cause: TerminatedInstance }
29562024-11-18T18:52:19.856ZWARNSledAgent (InstanceManager): Halting and removing zone: oxz_propolis-server_a69389c5-8d0a-47da-b47d-3c8aacfaa217 file = sled-agent/src/instance.rs:1032 instance_id = 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 propolis_id = a69389c5-8d0a-47da-b47d-3c8aacfaa217
29572024-11-18T18:52:19.864ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 834 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:39887 req_id = 4440d016-178d-4b99-b784-494a7afbb9c7 response_code = 204 uri = /vpc-routes
29582024-11-18T18:52:19.892ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:55312
29592024-11-18T18:52:19.892ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 251 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:55312 req_id = 405e21cb-7cb8-4d9e-8205-a23319f6852f response_code = 200 uri = /v2p
29602024-11-18T18:52:19.913ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 128108 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:44482 req_id = 16690902-8957-4a21-a71e-40ce2e6b5efc response_code = 204 uri = /switch-ports
29612024-11-18T18:52:19.914ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 177 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:44482 req_id = 2726ffb1-dc35-4d9e-b65e-0afa8d749d88 response_code = 200 uri = /network-bootstore-config
29622024-11-18T18:52:21.605ZINFOSledAgent (InstanceManager): halt_and_remove_logged: Previous zone state: Running file = illumos-utils/src/zone.rs:279 instance_id = 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 propolis_id = a69389c5-8d0a-47da-b47d-3c8aacfaa217
29632024-11-18T18:52:21.626ZINFOSledAgent (PortManager): Removed route set for subnet file = illumos-utils/src/opte/port_manager.rs:964 id = RouterId { vni: Vni(6772514), kind: System }
29642024-11-18T18:52:21.626ZINFOSledAgent (PortManager): Removed route set for subnet file = illumos-utils/src/opte/port_manager.rs:964 id = RouterId { vni: Vni(6772514), kind: Custom(V4(Ipv4Net { addr: 172.30.0.0, width: 22 })) }
29652024-11-18T18:52:21.626ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 propolis_id = a69389c5-8d0a-47da-b47d-3c8aacfaa217 state = SledVmmState { vmm_state: VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2024-11-18T18:52:21.626397461Z }, migration_in: None, migration_out: None }
29662024-11-18T18:52:21.785ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 propolis_id = a69389c5-8d0a-47da-b47d-3c8aacfaa217 state = SledVmmState { vmm_state: VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2024-11-18T18:52:21.626397461Z }, migration_in: None, migration_out: None }
29672024-11-18T18:52:21.789ZINFOSledAgent (InstanceManager): State monitoring task complete file = sled-agent/src/instance.rs:969 instance_id = 0072dd5d-b0a3-46f5-8e43-a2f71db9f5a6 propolis_id = a69389c5-8d0a-47da-b47d-3c8aacfaa217
29682024-11-18T18:52:21.841ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2281126 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:51571 req_id = 1b923bb1-c513-4d98-89a3-0e33be26b7de response_code = 200 uri = /vmms/a69389c5-8d0a-47da-b47d-3c8aacfaa217/state
29692024-11-18T18:52:21.841ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2208250 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:54541 req_id = 7680b4e7-f437-46e8-9c64-03db5d262ae6 response_code = 200 uri = /vmms/a69389c5-8d0a-47da-b47d-3c8aacfaa217/state
29702024-11-18T18:52:22.455ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:62348
29712024-11-18T18:52:22.455ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 97 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:62348 req_id = 66d93f14-1a2a-45ea-a330-2357576bea1f response_code = 200 uri = /vpc-routes
29722024-11-18T18:52:22.519ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
29732024-11-18T18:52:22.519ZINFOSledAgent: IGW mapping changed; external IPs refreshed file = sled-agent/src/sled_agent.rs:1231 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
29742024-11-18T18:52:22.519ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 159 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:62348 req_id = 11349acd-6be8-4f06-ba4b-d6ff39545c5a response_code = 204 uri = /eip-gateways
29752024-11-18T18:52:22.528ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 318 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:62348 req_id = 9dca42e2-cdea-4df2-8731-8386a541b29f response_code = 204 uri = /vpc-routes
29762024-11-18T18:52:22.646ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:54294
29772024-11-18T18:52:22.646ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 123 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:54294 req_id = 2d4302c7-55f0-4c97-9e67-add5c76290aa response_code = 200 uri = /v2p
29782024-11-18T18:52:24.019ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:40669
29792024-11-18T18:52:24.019ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 175 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:40669 req_id = 51227f99-96bd-4871-b63e-8e29d29e36c4 response_code = 200 uri = /v2p
29802024-11-18T18:52:24.189ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:52099
29812024-11-18T18:52:24.189ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 144 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:52099 req_id = fdd2c4d7-2516-4f03-858b-0fd10cf948b8 response_code = 200 uri = /vpc-routes
29822024-11-18T18:52:24.247ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
29832024-11-18T18:52:24.247ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 91 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:52099 req_id = aaac3c49-81aa-4b53-8f4f-489db3a142c1 response_code = 204 uri = /eip-gateways
29842024-11-18T18:52:24.256ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 201 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:52099 req_id = 49e9ed0f-3b6b-4f0a-8fcd-cc6797886663 response_code = 204 uri = /vpc-routes
29852024-11-18T18:52:24.865ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:50997
29862024-11-18T18:52:24.865ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 92 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:50997 req_id = ca91e805-57df-479f-99bf-ec7eeb93dab9 response_code = 200 uri = /vpc-routes
29872024-11-18T18:52:24.881ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2 values = ExternalIpGatewayMap { mappings: {} }
29882024-11-18T18:52:24.881ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 75 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:50997 req_id = d8fd8fe4-fa38-4b5d-9970-bd3e58f44850 response_code = 204 uri = /eip-gateways
29892024-11-18T18:52:24.890ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 159 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:50997 req_id = e2bf5e03-e70d-4c81-a60a-acbfc4ce2799 response_code = 204 uri = /vpc-routes
29902024-11-18T18:52:25.903ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
29912024-11-18T18:52:25.906ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
29922024-11-18T18:52:25.907ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2
29932024-11-18T18:52:25.957ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = ee2b3f44-b73f-4358-9df7-edb19f5079e2