Max level shown:
1 [ Nov 2 03:55:07 Enabled. ]
2 [ Nov 2 03:55:07 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/sled-agent/sled-agent run /opt/oxide/sled-agent/pkg/config.toml &"). ]
3 [ Nov 2 03:55:07 Method "start" exited with status 0. ]
4 [ Nov 2 03:55:07 Rereading configuration. ]
5 [ Nov 2 03:55:07 No 'refresh' method defined. Treating as :true. ]
6 note: configured to log to "/dev/stdout"
72024-11-02T03:55:07.172ZINFOSledAgent: Ensuring contract reaper thread file = illumos-utils/src/running_zone.rs:113
82024-11-02T03:55:07.172ZINFOSledAgent (BootstrapAgentStartup): finding links ["net0", "net1"] file = sled-agent/src/bootstrap/pre_server.rs:255
92024-11-02T03:55:07.224ZINFOSledAgent (BootstrapAgentStartup): Starting mg-ddm service addr-objs = [AddrObject { interface: "net0", name: "ll" }, AddrObject { interface: "net1", name: "ll" }] file = sled-agent/src/bootstrap/pre_server.rs:262
102024-11-02T03:55:07.224ZINFOSledAgent (BootstrapAgentStartup): Importing mg-ddm service file = sled-agent/src/bootstrap/maghemite.rs:42 path = /opt/oxide/mg-ddm/pkg/ddm/manifest.xml
112024-11-02T03:55:07.338ZINFOSledAgent (BootstrapAgentStartup): Setting mg-ddm interfaces file = sled-agent/src/bootstrap/maghemite.rs:50 interfaces = ("net0/ll" "net1/ll")
122024-11-02T03:55:07.347ZINFOSledAgent (BootstrapAgentStartup): Enabling mg-ddm service file = sled-agent/src/bootstrap/maghemite.rs:56
132024-11-02T03:55:07.354ZINFOSledAgent (BootstrapAgentStartup): Ensuring zfs key directory exists file = sled-agent/src/bootstrap/pre_server.rs:272 path = /var/run/oxide/
142024-11-02T03:55:07.955ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fdb0:18c0:4d0d:9fb2::, width: 64 }
152024-11-02T03:55:08.091ZINFOSledAgent: Starting KeyManager file = sled-agent/src/long_running_tasks.rs:126
162024-11-02T03:55:08.091ZINFOSledAgent: Starting StorageManager file = sled-agent/src/long_running_tasks.rs:138
172024-11-02T03:55:08.091ZINFOSledAgent: Starting StorageMonitor file = sled-agent/src/long_running_tasks.rs:151
182024-11-02T03:55:08.092ZINFOSledAgent: Starting HardwareManager file = sled-agent/src/long_running_tasks.rs:173 nongimlet_observed_disks = [] sled_mode = Scrimlet { asic: SoftNpuZone }
192024-11-02T03:55:08.092ZINFOSledAgent (HardwareManager): Creating HardwareManager file = sled-hardware/src/illumos/mod.rs:717
202024-11-02T03:55:08.092ZINFOSledAgent (DumpSetup-worker): DumpSetup poll loop started. file = sled-agent/src/dump_setup.rs:520
212024-11-02T03:55:08.095ZINFOSledAgent (HardwareManager): Generated i86pc baseboard Pc { identifier: "buskin", model: "i86pc" } file = sled-hardware/src/illumos/mod.rs:624
222024-11-02T03:55:08.095ZWARNSledAgent (HardwareManager): Device is not a Gimlet (i86pc), proceeding with null hardware view file = sled-hardware/src/illumos/mod.rs:766
232024-11-02T03:55:08.095ZINFOSledAgent: Starting HardwareMonitor file = sled-agent/src/long_running_tasks.rs:187
242024-11-02T03:55:08.095ZINFOSledAgent (HardwareManager): Monitoring for hardware updates file = sled-hardware/src/illumos/mod.rs:825
252024-11-02T03:55:08.095ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/m2_0.vdev
262024-11-02T03:55:08.095ZINFOSledAgent (HardwareMonitor): Checking current full hardware snapshot disks = {} file = sled-agent/src/hardware_monitor.rs:271 underlay_network_info = None
272024-11-02T03:55:08.095ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:451
282024-11-02T03:55:08.095ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
292024-11-02T03:55:08.095ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } path = /scratch/m2_0.vdev
302024-11-02T03:55:08.106ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/m2_0.vdev file = sled-hardware/src/disk.rs:367
312024-11-02T03:55:08.106ZINFOSledAgent (StorageResources): Formatting zpool with generated ID file = sled-hardware/src/disk.rs:378 id = 6308f1ec-c08e-4282-a1c2-db37d6b8a044 (zpool)
322024-11-02T03:55:08.203ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:181 zpool = ZpoolName { id: 6308f1ec-c08e-4282-a1c2-db37d6b8a044 (zpool), kind: Internal }
332024-11-02T03:55:08.670ZINFOSledAgent (StorageManager): Received DetectedRawDisksChanged { raw_disks: {}, tx: ... } file = sled-storage/src/manager.rs:451
342024-11-02T03:55:08.670ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/m2_1.vdev
352024-11-02T03:55:08.670ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
362024-11-02T03:55:08.670ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
372024-11-02T03:55:08.670ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:451
382024-11-02T03:55:08.670ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
392024-11-02T03:55:08.670ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" } path = /scratch/m2_1.vdev
402024-11-02T03:55:08.679ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/m2_1.vdev file = sled-hardware/src/disk.rs:367
412024-11-02T03:55:08.679ZINFOSledAgent (StorageResources): Formatting zpool with generated ID file = sled-hardware/src/disk.rs:378 id = fda73319-9c6d-44d8-9377-b69537374c59 (zpool)
422024-11-02T03:55:08.769ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:181 zpool = ZpoolName { id: fda73319-9c6d-44d8-9377-b69537374c59 (zpool), kind: Internal }
432024-11-02T03:55:09.247ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_0.vdev
442024-11-02T03:55:09.247ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 6308f1ec-c08e-4282-a1c2-db37d6b8a044 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: fda73319-9c6d-44d8-9377-b69537374c59 (zpool), kind: Internal } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
452024-11-02T03:55:09.247ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
462024-11-02T03:55:09.247ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
472024-11-02T03:55:09.247ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:451
482024-11-02T03:55:09.247ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
492024-11-02T03:55:09.247ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_1.vdev
502024-11-02T03:55:09.247ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 6308f1ec-c08e-4282-a1c2-db37d6b8a044 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: fda73319-9c6d-44d8-9377-b69537374c59 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
512024-11-02T03:55:09.247ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:451
522024-11-02T03:55:09.247ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
532024-11-02T03:55:09.247ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
542024-11-02T03:55:09.247ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
552024-11-02T03:55:09.247ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_2.vdev
562024-11-02T03:55:09.247ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:451
572024-11-02T03:55:09.248ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
582024-11-02T03:55:09.248ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 6308f1ec-c08e-4282-a1c2-db37d6b8a044 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: fda73319-9c6d-44d8-9377-b69537374c59 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
592024-11-02T03:55:09.248ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
602024-11-02T03:55:09.248ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
612024-11-02T03:55:09.248ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_3.vdev
622024-11-02T03:55:09.248ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 6308f1ec-c08e-4282-a1c2-db37d6b8a044 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: fda73319-9c6d-44d8-9377-b69537374c59 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
632024-11-02T03:55:09.248ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
642024-11-02T03:55:09.248ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
652024-11-02T03:55:09.248ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:451
662024-11-02T03:55:09.248ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
672024-11-02T03:55:09.248ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_4.vdev
682024-11-02T03:55:09.248ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:451
692024-11-02T03:55:09.248ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 6308f1ec-c08e-4282-a1c2-db37d6b8a044 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: fda73319-9c6d-44d8-9377-b69537374c59 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
702024-11-02T03:55:09.248ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
712024-11-02T03:55:09.248ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
722024-11-02T03:55:09.248ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
732024-11-02T03:55:09.248ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_5.vdev
742024-11-02T03:55:09.248ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 6308f1ec-c08e-4282-a1c2-db37d6b8a044 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: fda73319-9c6d-44d8-9377-b69537374c59 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
752024-11-02T03:55:09.248ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:451
762024-11-02T03:55:09.248ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
772024-11-02T03:55:09.248ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
782024-11-02T03:55:09.248ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
792024-11-02T03:55:09.248ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_6.vdev
802024-11-02T03:55:09.248ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 6308f1ec-c08e-4282-a1c2-db37d6b8a044 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: fda73319-9c6d-44d8-9377-b69537374c59 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
812024-11-02T03:55:09.248ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
822024-11-02T03:55:09.248ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:451
832024-11-02T03:55:09.248ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
842024-11-02T03:55:09.248ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
852024-11-02T03:55:09.248ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_7.vdev
862024-11-02T03:55:09.248ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:451
872024-11-02T03:55:09.248ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
882024-11-02T03:55:09.248ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 6308f1ec-c08e-4282-a1c2-db37d6b8a044 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: fda73319-9c6d-44d8-9377-b69537374c59 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
892024-11-02T03:55:09.248ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
902024-11-02T03:55:09.248ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
912024-11-02T03:55:09.248ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_8.vdev
922024-11-02T03:55:09.248ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 6308f1ec-c08e-4282-a1c2-db37d6b8a044 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: fda73319-9c6d-44d8-9377-b69537374c59 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
932024-11-02T03:55:09.248ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
942024-11-02T03:55:09.248ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
952024-11-02T03:55:09.248ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:451
962024-11-02T03:55:09.248ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
972024-11-02T03:55:09.248ZINFOSledAgent: Waiting for boot disk file = sled-agent/src/long_running_tasks.rs:97
982024-11-02T03:55:09.248ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 6308f1ec-c08e-4282-a1c2-db37d6b8a044 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: fda73319-9c6d-44d8-9377-b69537374c59 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
992024-11-02T03:55:09.248ZINFOSledAgent: Found boot disk DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } file = sled-agent/src/long_running_tasks.rs:99
1002024-11-02T03:55:09.248ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
1012024-11-02T03:55:09.248ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
1022024-11-02T03:55:09.248ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
1032024-11-02T03:55:09.248ZINFOSledAgent: Starting Bootstore file = sled-agent/src/long_running_tasks.rs:211
1042024-11-02T03:55:09.248ZINFOSledAgent (bootstore): No ledger in /pool/int/6308f1ec-c08e-4282-a1c2-db37d6b8a044/cluster/bootstore-fsm-state.json file = common/src/ledger.rs:188 peer_id = pc-buskin-i86pc
1052024-11-02T03:55:09.248ZINFOSledAgent (bootstore): No ledger in /pool/int/fda73319-9c6d-44d8-9377-b69537374c59/cluster/bootstore-fsm-state.json file = common/src/ledger.rs:188 peer_id = pc-buskin-i86pc
1062024-11-02T03:55:09.248ZINFOSledAgent (bootstore): No ledger found. Loading Fsm::State as Uninitialized file = bootstore/src/schemes/v0/storage.rs:83 peer_id = pc-buskin-i86pc
1072024-11-02T03:55:09.248ZINFOSledAgent (bootstore): No ledger in /pool/int/6308f1ec-c08e-4282-a1c2-db37d6b8a044/cluster/bootstore-network-config.json file = common/src/ledger.rs:188 peer_id = pc-buskin-i86pc
1082024-11-02T03:55:09.248ZINFOSledAgent (bootstore): No ledger in /pool/int/fda73319-9c6d-44d8-9377-b69537374c59/cluster/bootstore-network-config.json file = common/src/ledger.rs:188 peer_id = pc-buskin-i86pc
1092024-11-02T03:55:09.248ZINFOSledAgent (bootstore): No ledger found for network config file = bootstore/src/schemes/v0/storage.rs:146 peer_id = pc-buskin-i86pc
1102024-11-02T03:55:09.248ZINFOSledAgent: Starting Bootstore DDMD poller file = sled-agent/src/long_running_tasks.rs:216
1112024-11-02T03:55:09.248ZINFOSledAgent: Starting ZoneBundler related tasks file = sled-agent/src/long_running_tasks.rs:231
1122024-11-02T03:55:09.260ZINFOSledAgent (ServiceManager): Creating ServiceManager file = sled-agent/src/services.rs:733
1132024-11-02T03:55:09.260ZINFOSledAgent (auto-cleanup-task): top of bundle cleanup loop file = sled-agent/src/zone_bundle.rs:186 next_cleanup = Instant { tv_sec: 9813, tv_nsec: 282876144 } time_to_next_cleanup = 599.999914419s
1142024-11-02T03:55:09.260ZINFOSledAgent (ServiceManager): Ensuring scrimlet services (enabling services) file = sled-agent/src/services.rs:3909
1152024-11-02T03:55:09.260ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
1162024-11-02T03:55:09.260ZINFOSledAgent (BootstrapAgentStartup): No ledger in /pool/int/6308f1ec-c08e-4282-a1c2-db37d6b8a044/config/sled-agent-request.json file = common/src/ledger.rs:188
1172024-11-02T03:55:09.262ZINFOSledAgent (BootstrapAgentStartup): No ledger in /pool/int/fda73319-9c6d-44d8-9377-b69537374c59/config/sled-agent-request.json file = common/src/ledger.rs:188
1182024-11-02T03:55:09.263ZINFOSledAgent (dropshot (BootstrapAgent)): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:197 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80
1192024-11-02T03:55:09.264ZINFOSledAgent (SledAgentSprocketsServer): Started listening file = sled-agent/src/bootstrap/sprockets_server.rs:51 local_addr = [fdb0:18c0:4d0d:9fb2::1]:12346
1202024-11-02T03:55:09.265ZINFOSledAgent (RSS): Injecting RSS configuration: RackInitializeRequest { trust_quorum_peers: None, bootstrap_discovery: OnlyOurs, ntp_servers: [ "0.pool.ntp.org", ], dns_servers: [ 1.1.1.1, 9.9.9.9, ], internal_services_ip_pool_ranges: [ V4( Ipv4Range { first: 10.151.1.164, last: 10.151.1.173, }, ), ], external_dns_ips: [ 10.151.1.164, 10.151.1.165, ], external_dns_zone_name: "oxide.test", external_certificates: "<redacted>", recovery_silo: RecoverySiloConfig { silo_name: Name( "recovery", ), user_name: UserId( "recovery", ), user_password_hash: NewPasswordHash( PasswordHashString { string: "$argon2id$v=19$m=98304,t=13,p=1$RUlWc0ZxaHo0WFdrN0N6ZQ$S8p52j85GPvMhR/ek3GL0el/oProgTwWpHJZ8lsQQoY", encoding: B64, }, ), }, rack_network_config: RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, infra_ip_first: 10.151.1.174, infra_ip_last: 10.151.1.174, ports: [ PortConfigV2 { routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), nexthop: 10.151.1.100, vlan_id: None, rib_priority: None, }, ], addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.1.174, width: 24, }, ), vlan_id: None, }, ], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: None, bgp_peers: [], autoneg: false, lldp: None, tx_eq: None, }, ], bgp: [], bfd: [], }, allowed_source_ips: Any, } file = sled-agent/src/rack_setup/service.rs:1083
1212024-11-02T03:55:09.265ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = internal-dns/resolver/src/resolver.rs:111
1222024-11-02T03:55:09.265ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
1232024-11-02T03:55:09.265ZINFOSledAgent (RSS): No ledger in /pool/int/6308f1ec-c08e-4282-a1c2-db37d6b8a044/config/rss-plan-completed.marker file = common/src/ledger.rs:188
1242024-11-02T03:55:09.265ZINFOSledAgent (RSS): No ledger in /pool/int/fda73319-9c6d-44d8-9377-b69537374c59/config/rss-plan-completed.marker file = common/src/ledger.rs:188
1252024-11-02T03:55:09.265ZINFOSledAgent (RSS): RSS configuration has not been fully applied yet file = sled-agent/src/rack_setup/service.rs:1146
1262024-11-02T03:55:09.265ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
1272024-11-02T03:55:09.265ZINFOSledAgent (RSS): No ledger in /pool/int/6308f1ec-c08e-4282-a1c2-db37d6b8a044/config/rss-sled-plan.json file = common/src/ledger.rs:188
1282024-11-02T03:55:09.265ZINFOSledAgent (RSS): No ledger in /pool/int/fda73319-9c6d-44d8-9377-b69537374c59/config/rss-sled-plan.json file = common/src/ledger.rs:188
1292024-11-02T03:55:09.265ZINFOSledAgent (RSS): Creating new allocation plan file = sled-agent/src/rack_setup/service.rs:1188
1302024-11-02T03:55:09.265ZINFOSledAgent (RSS): Serializing plan file = sled-agent/src/rack_setup/plan/sled.rs:156
1312024-11-02T03:55:09.265ZINFOSledAgent (RSS): Creating plan for the sled at fdb0:18c0:4d0d:9fb2::1 file = sled-agent/src/rack_setup/plan/sled.rs:129
1322024-11-02T03:55:09.265ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
1332024-11-02T03:55:09.265ZINFOSledAgent (RSS): Writing ledger to /pool/int/6308f1ec-c08e-4282-a1c2-db37d6b8a044/config/.rss-sled-plan.json.tmp file = common/src/ledger.rs:199
1342024-11-02T03:55:09.265ZINFOSledAgent (RSS): Writing ledger to /pool/int/fda73319-9c6d-44d8-9377-b69537374c59/config/.rss-sled-plan.json.tmp file = common/src/ledger.rs:199
1352024-11-02T03:55:09.266ZINFOSledAgent (RSS): Sled plan written to storage: Plan { rack_id: e190f004-2376-4aae-8296-5a5757dc1123, sleds: { [fdb0:18c0:4d0d:9fb2::1]:12346: StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: 5b8cdd5a-f021-45f2-8323-b6c8f9776060 (sled), rack_id: e190f004-2376-4aae-8296-5a5757dc1123, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64, }, }, }, }, }, config: RackInitializeRequest { trust_quorum_peers: None, bootstrap_discovery: OnlyOurs, ntp_servers: [ "0.pool.ntp.org", ], dns_servers: [ 1.1.1.1, 9.9.9.9, ], internal_services_ip_pool_ranges: [ V4( Ipv4Range { first: 10.151.1.164, last: 10.151.1.173, }, ), ], external_dns_ips: [ 10.151.1.164, 10.151.1.165, ], external_dns_zone_name: "oxide.test", external_certificates: "<redacted>", recovery_silo: RecoverySiloConfig { silo_name: Name( "recovery", ), user_name: UserId( "recovery", ), user_password_hash: NewPasswordHash( PasswordHashString { string: "$argon2id$v=19$m=98304,t=13,p=1$RUlWc0ZxaHo0WFdrN0N6ZQ$S8p52j85GPvMhR/ek3GL0el/oProgTwWpHJZ8lsQQoY", encoding: B64, }, ), }, rack_network_config: RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, infra_ip_first: 10.151.1.174, infra_ip_last: 10.151.1.174, ports: [ PortConfigV2 { routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), nexthop: 10.151.1.100, vlan_id: None, rib_priority: None, }, ], addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.1.174, width: 24, }, ), vlan_id: None, }, ], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: None, bgp_peers: [], autoneg: false, lldp: None, tx_eq: None, }, ], bgp: [], bfd: [], }, allowed_source_ips: Any, }, } file = sled-agent/src/rack_setup/plan/sled.rs:176
1362024-11-02T03:55:09.266ZINFOSledAgent (RSS): Writing Rack Network Configuration to bootstore file = sled-agent/src/rack_setup/service.rs:1222
1372024-11-02T03:55:09.266ZINFOSledAgent (bootstore): Attempting to update network config with generation: 1, current_generation: 0 file = bootstore/src/schemes/v0/peer.rs:564 peer_id = pc-buskin-i86pc
1382024-11-02T03:55:09.266ZINFOSledAgent (bootstore): Writing ledger to /pool/int/6308f1ec-c08e-4282-a1c2-db37d6b8a044/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-buskin-i86pc
1392024-11-02T03:55:09.266ZINFOSledAgent (bootstore): Writing ledger to /pool/int/fda73319-9c6d-44d8-9377-b69537374c59/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-buskin-i86pc
1402024-11-02T03:55:09.266ZINFOSledAgent (bootstore): Broadcasting network config with generation 1 file = bootstore/src/schemes/v0/peer.rs:648 peer_id = pc-buskin-i86pc
1412024-11-02T03:55:09.266ZINFOSledAgent (BootstrapAgentRssHandler): Received initialization request from RSS file = sled-agent/src/bootstrap/rss_handle.rs:232 request = StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: 5b8cdd5a-f021-45f2-8323-b6c8f9776060 (sled), rack_id: e190f004-2376-4aae-8296-5a5757dc1123, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64 } } } } target_sled = [fdb0:18c0:4d0d:9fb2::1]:12346
1422024-11-02T03:55:09.266ZINFOSledAgent (ServiceManager): Enabling switch zone (new) file = sled-agent/src/services.rs:4206
1432024-11-02T03:55:09.266ZINFOSledAgent (ServiceManager): Starting switch zone file = sled-agent/src/services.rs:4588
1442024-11-02T03:55:09.271ZINFOSledAgent (SledAgentSprocketsClient): Loaded keys and certs BootstrapAgentClient = [fdb0:18c0:4d0d:9fb2::1]:12346 file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/client.rs:50
1452024-11-02T03:55:09.277ZINFOSledAgent (SledAgentSprocketsClient): Certificate chain verified successfully BootstrapAgentClient = [fdb0:18c0:4d0d:9fb2::1]:12346 file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/keys.rs:307
1462024-11-02T03:55:09.277ZINFOSledAgent (SledAgentSprocketsClient): Signature verified successfully BootstrapAgentClient = [fdb0:18c0:4d0d:9fb2::1]:12346 file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/keys.rs:376
1472024-11-02T03:55:09.281ZINFOSledAgent (SledAgentSprocketsServer): Certificate chain verified successfully file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/keys.rs:307
1482024-11-02T03:55:09.282ZINFOSledAgent (SledAgentSprocketsServer): Signature verified successfully file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/keys.rs:376
1492024-11-02T03:55:09.282ZINFOSledAgent (SledAgentSprocketsServer): Accepted connection file = sled-agent/src/bootstrap/sprockets_server.rs:74 remote_addr = [fdb0:18c0:4d0d:9fb2::1]:48127
1502024-11-02T03:55:09.282ZINFOSledAgent (SledAgentMain): Loading Sled Agent: StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: 5b8cdd5a-f021-45f2-8323-b6c8f9776060 (sled), rack_id: e190f004-2376-4aae-8296-5a5757dc1123, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64 } } } } file = sled-agent/src/bootstrap/server.rs:362
1512024-11-02T03:55:09.282ZINFOSledAgent (SledAgentMain): KeyManager: using hardcoded secret retriever file = sled-agent/src/bootstrap/server.rs:377
1522024-11-02T03:55:09.282ZINFOSledAgent: setting up sled agent server file = sled-agent/src/server.rs:44
1532024-11-02T03:55:09.282ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = internal-dns/resolver/src/resolver.rs:111
1542024-11-02T03:55:09.282ZINFOSledAgent (StorageManager): Received KeyManagerReady file = sled-storage/src/manager.rs:451
1552024-11-02T03:55:09.282ZINFOSledAgent (StorageManager): Loading storage config file = sled-storage/src/manager.rs:644
1562024-11-02T03:55:09.282ZINFOSledAgent (StorageManager): No ledger in /pool/int/6308f1ec-c08e-4282-a1c2-db37d6b8a044/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = load_disks_ledger
1572024-11-02T03:55:09.282ZINFOSledAgent (StorageManager): No ledger in /pool/int/fda73319-9c6d-44d8-9377-b69537374c59/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = load_disks_ledger
1582024-11-02T03:55:09.282ZINFOSledAgent (StorageManager): No ledger of physical disks exists file = sled-storage/src/manager.rs:633
1592024-11-02T03:55:09.282ZINFOSledAgent (StorageManager): KeyManager ready, but no ledger detected file = sled-storage/src/manager.rs:661
1602024-11-02T03:55:09.282ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:101::, width: 64 }
1612024-11-02T03:55:09.282ZINFOSledAgent (StorageManager): automatically managing disks file = sled-storage/src/manager.rs:440
1622024-11-02T03:55:09.282ZINFOSledAgent (StorageResources): Synchronizing disk managment file = sled-storage/src/resources.rs:302
1632024-11-02T03:55:09.282ZINFOSledAgent (StorageManager): Successfully synchronized disks without error file = sled-storage/src/manager.rs:532 result = DisksManagementResult { status: [] }
1642024-11-02T03:55:09.297ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
1652024-11-02T03:55:09.319ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_switch file = illumos-utils/src/zone.rs:325
1662024-11-02T03:55:09.336ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_switch file = illumos-utils/src/zone.rs:363
1672024-11-02T03:55:09.362ZINFOSledAgent: SledAgent::new(..) starting file = sled-agent/src/sled_agent.rs:408 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
1682024-11-02T03:55:09.362ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
1692024-11-02T03:55:09.362ZINFOSledAgent: Requested swap device of size 64 GiB file = sled-agent/src/sled_agent.rs:420 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
1702024-11-02T03:55:09.362ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1712024-11-02T03:55:09.366ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 87.619927ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "e92c9c89-45f7-4f58-9387-1af500f9aee0", "content-length": "124", "date": "Sat, 02 Nov 2024 03:55:09 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "e92c9c89-45f7-4f58-9387-1af500f9aee0" } file = clients/ddm-admin-client/src/lib.rs:177
1722024-11-02T03:55:09.375ZINFOSledAgent: attempting to create encrypted zvol: name="oxi_6308f1ec-c08e-4282-a1c2-db37d6b8a044/swap", size_gb=64 file = sled-agent/src/swap_device.rs:153
1732024-11-02T03:55:09.398ZINFOSledAgent: successfully created encrypted zvol: name="oxi_6308f1ec-c08e-4282-a1c2-db37d6b8a044/swap", size_gb=64 file = sled-agent/src/swap_device.rs:249
1742024-11-02T03:55:09.398ZINFOSledAgent: adding swap device: swapname="/dev/zvol/dsk/oxi_6308f1ec-c08e-4282-a1c2-db37d6b8a044/swap" file = sled-agent/src/swap_device.rs:96
1752024-11-02T03:55:09.404ZINFOSledAgent: Mounting backing filesystems file = sled-agent/src/sled_agent.rs:435 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
1762024-11-02T03:55:09.404ZINFOSledAgent (BackingFs): Processing fmd file = sled-agent/src/backing_fs.rs:128
1772024-11-02T03:55:09.404ZINFOSledAgent (BackingFs): Ensuring dataset oxi_6308f1ec-c08e-4282-a1c2-db37d6b8a044/backing/fmd file = sled-agent/src/backing_fs.rs:138
1782024-11-02T03:55:09.455ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1792024-11-02T03:55:09.455ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 153.797163ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "986555a0-c408-4ee4-8bae-b76a215c0239", "content-length": "124", "date": "Sat, 02 Nov 2024 03:55:09 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "986555a0-c408-4ee4-8bae-b76a215c0239" } file = clients/ddm-admin-client/src/lib.rs:177
1802024-11-02T03:55:09.489ZINFOSledAgent (BackingFs): Stopping service svc:/system/fmd:default file = sled-agent/src/backing_fs.rs:173
1812024-11-02T03:55:09.610ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1822024-11-02T03:55:09.611ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 130.769599ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "24a28a56-65a7-4764-9b9d-4ea1424be58e", "content-length": "124", "date": "Sat, 02 Nov 2024 03:55:09 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "24a28a56-65a7-4764-9b9d-4ea1424be58e" } file = clients/ddm-admin-client/src/lib.rs:177
1832024-11-02T03:55:09.708ZINFOSledAgent (BackingFs): Mounting oxi_6308f1ec-c08e-4282-a1c2-db37d6b8a044/backing/fmd on /var/fm/fmd file = sled-agent/src/backing_fs.rs:181
1842024-11-02T03:55:09.721ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/rsrc file = sled-agent/src/backing_fs.rs:189
1852024-11-02T03:55:09.721ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/ckpt file = sled-agent/src/backing_fs.rs:189
1862024-11-02T03:55:09.721ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/xprt file = sled-agent/src/backing_fs.rs:189
1872024-11-02T03:55:09.721ZINFOSledAgent (BackingFs): Starting service svc:/system/fmd:default file = sled-agent/src/backing_fs.rs:195
1882024-11-02T03:55:09.743ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1892024-11-02T03:55:09.743ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 187.378126ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "f1e22697-57c5-4bda-93f4-5bbc18d9f305", "content-length": "124", "date": "Sat, 02 Nov 2024 03:55:09 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "f1e22697-57c5-4bda-93f4-5bbc18d9f305" } file = clients/ddm-admin-client/src/lib.rs:177
1902024-11-02T03:55:09.932ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1912024-11-02T03:55:09.933ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 147.096371ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "b700f2d6-f3cc-4e13-bb03-23a78bd5c436", "content-length": "124", "date": "Sat, 02 Nov 2024 03:55:09 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "b700f2d6-f3cc-4e13-bb03-23a78bd5c436" } file = clients/ddm-admin-client/src/lib.rs:177
1922024-11-02T03:55:10.081ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1932024-11-02T03:55:10.081ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 337.650213ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "13c59c6d-245e-4b28-a957-31d8b89f1859", "content-length": "124", "date": "Sat, 02 Nov 2024 03:55:09 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "13c59c6d-245e-4b28-a957-31d8b89f1859" } file = clients/ddm-admin-client/src/lib.rs:177
1942024-11-02T03:55:10.421ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1952024-11-02T03:55:10.452ZINFOSledAgent: using '[AddrObject { interface: "net0", name: "ll" }, AddrObject { interface: "net1", name: "ll" }]' as data links for xde driver file = illumos-utils/src/opte/illumos.rs:93 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
1962024-11-02T03:55:10.457ZINFOSledAgent (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:197 local_addr = [fd00:1122:3344:101::1]:56897 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
1972024-11-02T03:55:10.457ZINFOSledAgent (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = internal-dns/resolver/src/resolver.rs:111 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
1982024-11-02T03:55:10.457ZINFOSledAgent (producer-server): starting oximeter metric producer server address = [fd00:1122:3344:101::1]:56897 file = oximeter/producer/src/lib.rs:283 interval = 30s producer_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
1992024-11-02T03:55:10.458ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 218.402072ms error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
2002024-11-02T03:55:10.459ZINFOSledAgent (VmmReservoirManager): 50% of 102952026112 physical ram = 51476013056 bytes) file = sled-agent/src/vmm_reservoir.rs:267 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
2012024-11-02T03:55:10.459ZINFOSledAgent (VmmReservoirManager): Setting reservoir size to 49090 MiB bytes file = sled-agent/src/vmm_reservoir.rs:275 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
2022024-11-02T03:55:10.459ZINFOSledAgent (ServiceManager): sled agent started file = sled-agent/src/services.rs:917 underlay_address = fd00:1122:3344:101::1
2032024-11-02T03:55:10.459ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = internal-dns/resolver/src/resolver.rs:111
2042024-11-02T03:55:10.679ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 309.123471ms error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
2052024-11-02T03:55:10.989ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 1.169674388s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
2062024-11-02T03:55:12.160ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 2.120751501s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
2072024-11-02T03:55:12.921ZINFOSledAgent (ServiceManager): Setting up dendrite service file = sled-agent/src/services.rs:2664
2082024-11-02T03:55:12.921ZINFOSledAgent (ServiceManager): Setting up lldpd service file = sled-agent/src/services.rs:2882
2092024-11-02T03:55:12.921ZINFOSledAgent (ServiceManager): Setting up MGS service file = sled-agent/src/services.rs:2534
2102024-11-02T03:55:12.921ZINFOSledAgent (ServiceManager): Setting up wicketd service file = sled-agent/src/services.rs:2581
2112024-11-02T03:55:12.921ZINFOSledAgent (ServiceManager): Setting up mgd service file = sled-agent/src/services.rs:2963
2122024-11-02T03:55:12.921ZINFOSledAgent (ServiceManager): Setting up mg-ddm service file = sled-agent/src/services.rs:3005
2132024-11-02T03:55:12.923ZINFOSledAgent (ServiceManager): Setting up tfport service file = sled-agent/src/services.rs:2807
2142024-11-02T03:55:12.924ZINFOSledAgent (ServiceManager): Setting up Simulated SP service file = sled-agent/src/services.rs:2575
2152024-11-02T03:55:12.924ZINFOSledAgent (ServiceManager): Profile for oxz_switch: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService0'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/mgs"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="address" value='[::1]:12225'/> <propval type="astring" name="id" value='1d565c44-21ca-41c0-b1d4-bcace41c9902'/> <propval type="astring" name="rack_id" value='e190f004-2376-4aae-8296-5a5757dc1123'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/wicketd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="address" value='[::1]:12226'/> <propval type="astring" name="artifact-address" value='[fdb0:18c0:4d0d:9fb2::2]:12227'/> <propval type="astring" name="baseboard-file" value='/opt/oxide/baseboard.json'/> <propval type="astring" name="mgs-address" value='[::1]:12225'/> <propval type="astring" name="nexus-proxy-address" value='[::]:12229'/> <propval type="astring" name="rack-subnet" value='fd00:1122:3344::'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/switch_zone_setup"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="gz_local_link_addr" value='fe80::8:20ff:fe27:43ab'/> <property type="astring" name="link_local_links"> <astring_list> <value_node value="tfportrear0_0" /> <value_node value="oxBootstrap0" /> </astring_list> </property> <propval type="astring" name="bootstrap_addr" value='fdb0:18c0:4d0d:9fb2::2'/> <propval type="astring" name="bootstrap_vnic" value='oxBootstrap0'/> <propval type="astring" name="baseboard_info" value='{"type":"pc","identifier":"buskin","model":"i86pc"}'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/dendrite"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="sled_id" value='5b8cdd5a-f021-45f2-8323-b6c8f9776060'/> <propval type="astring" name="rack_id" value='e190f004-2376-4aae-8296-5a5757dc1123'/> <propval type="astring" name="address" value='[::1]:12224'/> <propval type="astring" name="front_ports" value='1'/> <propval type="astring" name="rear_ports" value='1'/> <propval type="astring" name="port_config" value='/opt/oxide/dendrite/misc/softnpu_single_sled_config.toml'/> <propval type="astring" name="mgmt" value='uds'/> <propval type="astring" name="uds_path" value='/opt/softnpu/stuff'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/tfport"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="host" value='[::1]'/> <propval type="astring" name="port" value='12224'/> <propval type="astring" name="flags" value='--sync-only'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/lldpd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="board_rev" value='softnpu_front_1_rear_1'/> <propval type="astring" name="scrimlet_id" value='buskin'/> <propval type="astring" name="scrimlet_model" value='i86pc'/> <propval type="astring" name="address" value='[::1]:12230'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/pumpkind"> </service> <service version="1" type="service" name="oxide/mgd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="sled_uuid" value='5b8cdd5a-f021-45f2-8323-b6c8f9776060'/> <propval type="astring" name="rack_uuid" value='e190f004-2376-4aae-8296-5a5757dc1123'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/mg-ddm"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="mode" value='transit'/> <propval type="astring" name="dendrite" value='true'/> <propval type="astring" name="sled_uuid" value='5b8cdd5a-f021-45f2-8323-b6c8f9776060'/> <propval type="astring" name="rack_uuid" value='e190f004-2376-4aae-8296-5a5757dc1123'/> <propval type="astring" name="interfaces" value='tfportrear0_0/ll'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/uplink"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
2162024-11-02T03:55:12.924ZINFOSledAgent (ServiceManager): Booting oxz_switch zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_switch
2172024-11-02T03:55:13.279ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 34.630517ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2182024-11-02T03:55:13.484ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 73.770033ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2192024-11-02T03:55:13.687ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 215.853764ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2202024-11-02T03:55:14.078ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 396.294332ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2212024-11-02T03:55:14.282ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 3.281109765s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
2222024-11-02T03:55:14.508ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 908.329406ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2232024-11-02T03:55:15.529ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 673.437974ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2242024-11-02T03:55:16.393ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 891.795189ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2252024-11-02T03:55:17.469ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 665.582211ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2262024-11-02T03:55:17.566ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 10.898022528s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
2272024-11-02T03:55:18.146ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 985.076795ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2282024-11-02T03:55:19.518ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/5b8cdd5a-f021-45f2-8323-b6c8f9776060) file = sled-agent/src/nexus.rs:527 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
2292024-11-02T03:55:19.548ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
2302024-11-02T03:55:19.548ZINFOSledAgent (dropshot (Repo Depot)): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:197 local_addr = [fd00:1122:3344:101::1]:12348 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
2312024-11-02T03:55:19.549ZINFOSledAgent (dropshot (SledAgent)): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:197 local_addr = [fd00:1122:3344:101::1]:12345
2322024-11-02T03:55:19.549ZINFOSledAgent (SledAgentMain): Sled Agent loaded; recording configuration file = sled-agent/src/bootstrap/server.rs:432
2332024-11-02T03:55:19.549ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
2342024-11-02T03:55:19.549ZINFOSledAgent (SledAgentMain): Writing ledger to /pool/int/6308f1ec-c08e-4282-a1c2-db37d6b8a044/config/.sled-agent-request.json.tmp file = common/src/ledger.rs:199
2352024-11-02T03:55:19.549ZINFOSledAgent (SledAgentMain): Writing ledger to /pool/int/fda73319-9c6d-44d8-9377-b69537374c59/config/.sled-agent-request.json.tmp file = common/src/ledger.rs:199
2362024-11-02T03:55:19.550ZINFOSledAgent (HardwareMonitor): Sled Agent Started file = sled-agent/src/hardware_monitor.rs:138
2372024-11-02T03:55:19.550ZINFOSledAgent (HardwareMonitor): Checking current full hardware snapshot disks = {} file = sled-agent/src/hardware_monitor.rs:271 underlay_network_info = Some((fd00:1122:3344:101::2, Some(RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56 }, infra_ip_first: 10.151.1.174, infra_ip_last: 10.151.1.174, ports: [PortConfigV2 { routes: [RouteConfig { destination: V4(Ipv4Net { addr: 0.0.0.0, width: 0 }), nexthop: 10.151.1.100, vlan_id: None, rib_priority: None }], addresses: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: None, bgp_peers: [], autoneg: false, lldp: None, tx_eq: None }], bgp: [], bfd: [] })))
2382024-11-02T03:55:19.550ZINFOSledAgent (ServiceManager): Ensuring scrimlet services (enabling services) file = sled-agent/src/services.rs:3909
2392024-11-02T03:55:19.550ZINFOSledAgent (SledAgentSprocketsServer): Connection closed file = sled-agent/src/bootstrap/sprockets_server.rs:81 remote_addr = [fdb0:18c0:4d0d:9fb2::1]:48127
2402024-11-02T03:55:19.736ZINFOSledAgent (BootstrapAgentRssHandler): Peer agent initialized file = sled-agent/src/bootstrap/rss_handle.rs:117 peer = [fdb0:18c0:4d0d:9fb2::1]:12346
2412024-11-02T03:55:19.736ZINFOSledAgent (BootstrapAgentRssHandler): Initialized sled agent file = sled-agent/src/bootstrap/rss_handle.rs:252 target_sled = [fdb0:18c0:4d0d:9fb2::1]:12346
2422024-11-02T03:55:19.736ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
2432024-11-02T03:55:19.736ZINFOSledAgent (RSS): No ledger in /pool/int/6308f1ec-c08e-4282-a1c2-db37d6b8a044/config/rss-service-plan-v5.json file = common/src/ledger.rs:188
2442024-11-02T03:55:19.736ZINFOSledAgent (RSS): No ledger in /pool/int/fda73319-9c6d-44d8-9377-b69537374c59/config/rss-service-plan-v5.json file = common/src/ledger.rs:188
2452024-11-02T03:55:19.736ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
2462024-11-02T03:55:19.736ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
2472024-11-02T03:55:19.736ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
2482024-11-02T03:55:19.736ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
2492024-11-02T03:55:19.740ZINFOSledAgent (ServiceManager): Re-enabling running switch zone (new address) file = sled-agent/src/services.rs:4230 new = [fd00:1122:3344:101::2, ::1] old = [::1]
2502024-11-02T03:55:19.740ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::2 exists file = sled-agent/src/services.rs:4246
2512024-11-02T03:55:19.740ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:101::2, prefix: 64 })) file = illumos-utils/src/running_zone.rs:534 zone = oxz_switch
2522024-11-02T03:55:20.012ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:41047
2532024-11-02T03:55:20.012ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
2542024-11-02T03:55:20.012ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
2552024-11-02T03:55:20.013ZINFOSledAgent (ServiceManager): No ledger in /pool/int/6308f1ec-c08e-4282-a1c2-db37d6b8a044/config/omicron-zones.json file = common/src/ledger.rs:188
2562024-11-02T03:55:20.013ZINFOSledAgent (ServiceManager): No ledger in /pool/int/fda73319-9c6d-44d8-9377-b69537374c59/config/omicron-zones.json file = common/src/ledger.rs:188
2572024-11-02T03:55:20.013ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 349 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:41047 req_id = 277d7850-3462-4d35-bd63-cc1f907f7ec9 response_code = 200 uri = /inventory
2582024-11-02T03:55:20.043ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:45899
2592024-11-02T03:55:20.043ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 38 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:45899 req_id = 8b4a582c-a097-4799-9ec5-73456d669b2c response_code = 200 uri = /sled-role
2602024-11-02T03:55:20.045ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
2612024-11-02T03:55:20.045ZINFOSledAgent (RSS): Writing ledger to /pool/int/6308f1ec-c08e-4282-a1c2-db37d6b8a044/config/.rss-service-plan-v5.json.tmp file = common/src/ledger.rs:199
2622024-11-02T03:55:20.045ZINFOSledAgent (RSS): Writing ledger to /pool/int/fda73319-9c6d-44d8-9377-b69537374c59/config/.rss-service-plan-v5.json.tmp file = common/src/ledger.rs:199
2632024-11-02T03:55:20.046ZINFOSledAgent (RSS): Service plan written to storage file = sled-agent/src/rack_setup/plan/service.rs:886
2642024-11-02T03:55:20.077ZINFOSledAgent (RSS): attempting to set up sled's storage: OmicronPhysicalDisksConfig { generation: Generation(1), disks: [OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, id: 8526f520-a18b-44d7-a827-a48c889beb4c, pool_id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, id: 14a3f979-6522-4a56-bb6b-6b85163b765d, pool_id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, id: af6d20f3-8542-4b9f-9aeb-a7759dd7ebec, pool_id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, id: 52b238cc-7012-44d0-bd18-bb349b7fb122, pool_id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, id: 25e773a7-f1e2-414c-b52d-63838100741b, pool_id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, id: adef6d7a-9cfd-4e96-8ca5-9d2a561e0a6e, pool_id: 39aca50d-85fd-4ecc-a443-45b121ccfb9a (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, id: 397c30cd-e781-4226-8d70-7f4ae079330c, pool_id: 6b0637e8-562e-46b1-92a8-a7478ef38477 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, id: 80805b37-c686-4a3b-98be-cd98923e6b76, pool_id: 010339bd-18fb-4f1c-9455-7caf9e67e152 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, id: 96dfd8af-1920-49a7-b947-e12f11ecc0c6, pool_id: e3d9cf44-b37a-4529-b1ca-03b2819010d3 (zpool) }] } file = sled-agent/src/rack_setup/service.rs:376 sled_address = [fd00:1122:3344:101::1]:12345
2652024-11-02T03:55:20.077ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:64176
2662024-11-02T03:55:20.077ZINFOSledAgent: physical disks ensure file = sled-agent/src/sled_agent.rs:886 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
2672024-11-02T03:55:20.077ZINFOSledAgent (StorageManager): Received OmicronPhysicalDisksEnsure { config: OmicronPhysicalDisksConfig { generation: Generation(1), disks: [OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, id: 8526f520-a18b-44d7-a827-a48c889beb4c, pool_id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, id: 14a3f979-6522-4a56-bb6b-6b85163b765d, pool_id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, id: af6d20f3-8542-4b9f-9aeb-a7759dd7ebec, pool_id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, id: 52b238cc-7012-44d0-bd18-bb349b7fb122, pool_id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, id: 25e773a7-f1e2-414c-b52d-63838100741b, pool_id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, id: adef6d7a-9cfd-4e96-8ca5-9d2a561e0a6e, pool_id: 39aca50d-85fd-4ecc-a443-45b121ccfb9a (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, id: 397c30cd-e781-4226-8d70-7f4ae079330c, pool_id: 6b0637e8-562e-46b1-92a8-a7478ef38477 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, id: 80805b37-c686-4a3b-98be-cd98923e6b76, pool_id: 010339bd-18fb-4f1c-9455-7caf9e67e152 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, id: 96dfd8af-1920-49a7-b947-e12f11ecc0c6, pool_id: e3d9cf44-b37a-4529-b1ca-03b2819010d3 (zpool) }] }, tx: ... } file = sled-storage/src/manager.rs:451
2682024-11-02T03:55:20.077ZINFOSledAgent (StorageManager): No ledger in /pool/int/6308f1ec-c08e-4282-a1c2-db37d6b8a044/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = omicron_physical_disks_ensure
2692024-11-02T03:55:20.077ZINFOSledAgent (StorageManager): No ledger in /pool/int/fda73319-9c6d-44d8-9377-b69537374c59/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = omicron_physical_disks_ensure
2702024-11-02T03:55:20.077ZINFOSledAgent (StorageManager): No previously-stored 'requested disks', creating new ledger file = sled-storage/src/manager.rs:866 request = omicron_physical_disks_ensure
2712024-11-02T03:55:20.077ZINFOSledAgent (StorageResources): Synchronizing disk managment file = sled-storage/src/resources.rs:302
2722024-11-02T03:55:20.077ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:342
2732024-11-02T03:55:20.077ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
2742024-11-02T03:55:20.077ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } path = /scratch/u2_0.vdev
2752024-11-02T03:55:20.200ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_0.vdev file = sled-hardware/src/disk.rs:367
2762024-11-02T03:55:20.200ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool)
2772024-11-02T03:55:20.638ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 3855537906354135706 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
2782024-11-02T03:55:20.638ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 11546169058853415843 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
2792024-11-02T03:55:20.639ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/5b8cdd5a-f021-45f2-8323-b6c8f9776060) file = sled-agent/src/probe_manager.rs:231 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
2802024-11-02T03:55:20.836ZINFOSledAgent (VmmReservoirManager): Finished setting reservoir size to 49090 MiB bytes file = sled-agent/src/vmm_reservoir.rs:279 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
2812024-11-02T03:55:20.883ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:181 zpool = ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }
2822024-11-02T03:55:20.891ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:225
2832024-11-02T03:55:20.891ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:227
2842024-11-02T03:55:20.891ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:231
2852024-11-02T03:55:20.891ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:233
2862024-11-02T03:55:20.891ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key")
2872024-11-02T03:55:20.891ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key")
2882024-11-02T03:55:20.891ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:245
2892024-11-02T03:55:20.928ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
2902024-11-02T03:55:20.936ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_714116db-1daa-4c0c-b42f-cb5b471285ca/crypt/zone file = sled-storage/src/dataset.rs:288
2912024-11-02T03:55:21.107ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_714116db-1daa-4c0c-b42f-cb5b471285ca file = sled-storage/src/dataset.rs:374
2922024-11-02T03:55:21.115ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:440
2932024-11-02T03:55:21.115ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:357
2942024-11-02T03:55:21.115ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:342
2952024-11-02T03:55:21.115ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
2962024-11-02T03:55:21.115ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } path = /scratch/u2_1.vdev
2972024-11-02T03:55:21.124ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_1.vdev file = sled-hardware/src/disk.rs:367
2982024-11-02T03:55:21.124ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool)
2992024-11-02T03:55:21.215ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:181 zpool = ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }
3002024-11-02T03:55:21.223ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:225
3012024-11-02T03:55:21.223ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:227
3022024-11-02T03:55:21.223ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:231
3032024-11-02T03:55:21.223ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:233
3042024-11-02T03:55:21.223ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key")
3052024-11-02T03:55:21.223ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key")
3062024-11-02T03:55:21.223ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:245
3072024-11-02T03:55:21.258ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3082024-11-02T03:55:21.266ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_68ab7f9e-24f1-481a-b99f-af893cdd74a3/crypt/zone file = sled-storage/src/dataset.rs:288
3092024-11-02T03:55:21.439ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_68ab7f9e-24f1-481a-b99f-af893cdd74a3 file = sled-storage/src/dataset.rs:374
3102024-11-02T03:55:21.446ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:440
3112024-11-02T03:55:21.446ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:357
3122024-11-02T03:55:21.446ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:342
3132024-11-02T03:55:21.446ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3142024-11-02T03:55:21.446ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } path = /scratch/u2_2.vdev
3152024-11-02T03:55:21.456ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_2.vdev file = sled-hardware/src/disk.rs:367
3162024-11-02T03:55:21.456ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool)
3172024-11-02T03:55:21.547ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/dataset.rs:181 zpool = ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }
3182024-11-02T03:55:21.555ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/dataset.rs:225
3192024-11-02T03:55:21.555ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:227
3202024-11-02T03:55:21.555ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:231
3212024-11-02T03:55:21.555ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:233
3222024-11-02T03:55:21.555ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key")
3232024-11-02T03:55:21.555ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key")
3242024-11-02T03:55:21.555ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:245
3252024-11-02T03:55:21.591ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3262024-11-02T03:55:21.600ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_4846b5cc-cb2b-4eca-94f1-57e5f623f3d0/crypt/zone file = sled-storage/src/dataset.rs:288
3272024-11-02T03:55:21.642ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/5b8cdd5a-f021-45f2-8323-b6c8f9776060) file = sled-agent/src/probe_manager.rs:231 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
3282024-11-02T03:55:21.774ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 file = sled-storage/src/dataset.rs:374
3292024-11-02T03:55:21.782ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:440
3302024-11-02T03:55:21.782ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:357
3312024-11-02T03:55:21.782ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:342
3322024-11-02T03:55:21.782ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3332024-11-02T03:55:21.782ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } path = /scratch/u2_3.vdev
3342024-11-02T03:55:21.792ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_3.vdev file = sled-hardware/src/disk.rs:367
3352024-11-02T03:55:21.792ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool)
3362024-11-02T03:55:21.887ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/dataset.rs:181 zpool = ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External }
3372024-11-02T03:55:21.895ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/dataset.rs:225
3382024-11-02T03:55:21.895ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:227
3392024-11-02T03:55:21.895ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:231
3402024-11-02T03:55:21.895ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:233
3412024-11-02T03:55:21.895ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key")
3422024-11-02T03:55:21.895ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key")
3432024-11-02T03:55:21.895ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:245
3442024-11-02T03:55:21.933ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3452024-11-02T03:55:21.941ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_61fda18b-b0fc-46cf-91c9-706f41acfc67/crypt/zone file = sled-storage/src/dataset.rs:288
3462024-11-02T03:55:22.115ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_61fda18b-b0fc-46cf-91c9-706f41acfc67 file = sled-storage/src/dataset.rs:374
3472024-11-02T03:55:22.122ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:440
3482024-11-02T03:55:22.122ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:357
3492024-11-02T03:55:22.122ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:342
3502024-11-02T03:55:22.122ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3512024-11-02T03:55:22.122ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } path = /scratch/u2_4.vdev
3522024-11-02T03:55:22.132ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_4.vdev file = sled-hardware/src/disk.rs:367
3532024-11-02T03:55:22.132ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool)
3542024-11-02T03:55:22.221ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/dataset.rs:181 zpool = ZpoolName { id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool), kind: External }
3552024-11-02T03:55:22.229ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/dataset.rs:225
3562024-11-02T03:55:22.229ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:227
3572024-11-02T03:55:22.229ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:231
3582024-11-02T03:55:22.229ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:233
3592024-11-02T03:55:22.229ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key")
3602024-11-02T03:55:22.229ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key")
3612024-11-02T03:55:22.229ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:245
3622024-11-02T03:55:22.265ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3632024-11-02T03:55:22.273ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_09ce30cf-7836-438d-8cfe-4980ee894ac4/crypt/zone file = sled-storage/src/dataset.rs:288
3642024-11-02T03:55:22.448ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_09ce30cf-7836-438d-8cfe-4980ee894ac4 file = sled-storage/src/dataset.rs:374
3652024-11-02T03:55:22.455ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:440
3662024-11-02T03:55:22.456ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:357
3672024-11-02T03:55:22.456ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:342
3682024-11-02T03:55:22.456ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3692024-11-02T03:55:22.456ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } path = /scratch/u2_5.vdev
3702024-11-02T03:55:22.465ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_5.vdev file = sled-hardware/src/disk.rs:367
3712024-11-02T03:55:22.465ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 39aca50d-85fd-4ecc-a443-45b121ccfb9a (zpool)
3722024-11-02T03:55:22.556ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/dataset.rs:181 zpool = ZpoolName { id: 39aca50d-85fd-4ecc-a443-45b121ccfb9a (zpool), kind: External }
3732024-11-02T03:55:22.564ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/dataset.rs:225
3742024-11-02T03:55:22.564ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:227
3752024-11-02T03:55:22.564ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:231
3762024-11-02T03:55:22.564ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:233
3772024-11-02T03:55:22.564ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key")
3782024-11-02T03:55:22.564ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key")
3792024-11-02T03:55:22.564ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:245
3802024-11-02T03:55:22.597ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3812024-11-02T03:55:22.604ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_39aca50d-85fd-4ecc-a443-45b121ccfb9a/crypt/zone file = sled-storage/src/dataset.rs:288
3822024-11-02T03:55:22.648ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/5b8cdd5a-f021-45f2-8323-b6c8f9776060) file = sled-agent/src/probe_manager.rs:231 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
3832024-11-02T03:55:22.785ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_39aca50d-85fd-4ecc-a443-45b121ccfb9a file = sled-storage/src/dataset.rs:374
3842024-11-02T03:55:22.793ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:440
3852024-11-02T03:55:22.793ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:357
3862024-11-02T03:55:22.793ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:342
3872024-11-02T03:55:22.793ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3882024-11-02T03:55:22.793ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } path = /scratch/u2_6.vdev
3892024-11-02T03:55:22.802ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_6.vdev file = sled-hardware/src/disk.rs:367
3902024-11-02T03:55:22.802ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 6b0637e8-562e-46b1-92a8-a7478ef38477 (zpool)
3912024-11-02T03:55:22.899ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/dataset.rs:181 zpool = ZpoolName { id: 6b0637e8-562e-46b1-92a8-a7478ef38477 (zpool), kind: External }
3922024-11-02T03:55:22.907ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/dataset.rs:225
3932024-11-02T03:55:22.907ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:227
3942024-11-02T03:55:22.907ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:231
3952024-11-02T03:55:22.907ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:233
3962024-11-02T03:55:22.907ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key")
3972024-11-02T03:55:22.907ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key")
3982024-11-02T03:55:22.907ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:245
3992024-11-02T03:55:22.943ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
4002024-11-02T03:55:22.951ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_6b0637e8-562e-46b1-92a8-a7478ef38477/crypt/zone file = sled-storage/src/dataset.rs:288
4012024-11-02T03:55:23.123ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_6b0637e8-562e-46b1-92a8-a7478ef38477 file = sled-storage/src/dataset.rs:374
4022024-11-02T03:55:23.132ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:440
4032024-11-02T03:55:23.132ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:357
4042024-11-02T03:55:23.132ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:342
4052024-11-02T03:55:23.132ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
4062024-11-02T03:55:23.132ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } path = /scratch/u2_7.vdev
4072024-11-02T03:55:23.142ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_7.vdev file = sled-hardware/src/disk.rs:367
4082024-11-02T03:55:23.142ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 010339bd-18fb-4f1c-9455-7caf9e67e152 (zpool)
4092024-11-02T03:55:23.236ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/dataset.rs:181 zpool = ZpoolName { id: 010339bd-18fb-4f1c-9455-7caf9e67e152 (zpool), kind: External }
4102024-11-02T03:55:23.244ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/dataset.rs:225
4112024-11-02T03:55:23.244ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:227
4122024-11-02T03:55:23.244ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:231
4132024-11-02T03:55:23.244ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:233
4142024-11-02T03:55:23.244ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key")
4152024-11-02T03:55:23.244ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key")
4162024-11-02T03:55:23.244ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:245
4172024-11-02T03:55:23.280ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
4182024-11-02T03:55:23.288ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_010339bd-18fb-4f1c-9455-7caf9e67e152/crypt/zone file = sled-storage/src/dataset.rs:288
4192024-11-02T03:55:23.461ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_010339bd-18fb-4f1c-9455-7caf9e67e152 file = sled-storage/src/dataset.rs:374
4202024-11-02T03:55:23.469ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:440
4212024-11-02T03:55:23.469ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:357
4222024-11-02T03:55:23.469ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:342
4232024-11-02T03:55:23.469ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
4242024-11-02T03:55:23.469ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } path = /scratch/u2_8.vdev
4252024-11-02T03:55:23.479ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_8.vdev file = sled-hardware/src/disk.rs:367
4262024-11-02T03:55:23.479ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = e3d9cf44-b37a-4529-b1ca-03b2819010d3 (zpool)
4272024-11-02T03:55:23.575ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/dataset.rs:181 zpool = ZpoolName { id: e3d9cf44-b37a-4529-b1ca-03b2819010d3 (zpool), kind: External }
4282024-11-02T03:55:23.583ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/dataset.rs:225
4292024-11-02T03:55:23.583ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:227
4302024-11-02T03:55:23.583ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:231
4312024-11-02T03:55:23.583ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:233
4322024-11-02T03:55:23.583ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key")
4332024-11-02T03:55:23.583ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key")
4342024-11-02T03:55:23.583ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:245
4352024-11-02T03:55:23.618ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
4362024-11-02T03:55:23.625ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_e3d9cf44-b37a-4529-b1ca-03b2819010d3/crypt/zone file = sled-storage/src/dataset.rs:288
4372024-11-02T03:55:23.651ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/5b8cdd5a-f021-45f2-8323-b6c8f9776060) file = sled-agent/src/probe_manager.rs:231 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
4382024-11-02T03:55:23.797ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_e3d9cf44-b37a-4529-b1ca-03b2819010d3 file = sled-storage/src/dataset.rs:374
4392024-11-02T03:55:23.805ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:440
4402024-11-02T03:55:23.805ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:357
4412024-11-02T03:55:23.805ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/6308f1ec-c08e-4282-a1c2-db37d6b8a044/config/.omicron-physical-disks.json.tmp file = common/src/ledger.rs:199 request = omicron_physical_disks_ensure
4422024-11-02T03:55:23.805ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 6308f1ec-c08e-4282-a1c2-db37d6b8a044 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: fda73319-9c6d-44d8-9377-b69537374c59 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 39aca50d-85fd-4ecc-a443-45b121ccfb9a (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 6b0637e8-562e-46b1-92a8-a7478ef38477 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 010339bd-18fb-4f1c-9455-7caf9e67e152 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: e3d9cf44-b37a-4529-b1ca-03b2819010d3 (zpool), kind: External } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
4432024-11-02T03:55:23.805ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
4442024-11-02T03:55:23.805ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
4452024-11-02T03:55:23.805ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/fda73319-9c6d-44d8-9377-b69537374c59/config/.omicron-physical-disks.json.tmp file = common/src/ledger.rs:199 request = omicron_physical_disks_ensure
4462024-11-02T03:55:23.806ZINFOSledAgent: physical disks ensure: Updated storage file = sled-agent/src/sled_agent.rs:890 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
4472024-11-02T03:55:23.806ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
4482024-11-02T03:55:23.806ZINFOSledAgent: physical disks ensure: Propagating new generation of disks file = sled-agent/src/sled_agent.rs:916 generation = Generation(1) sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
4492024-11-02T03:55:23.806ZINFOSledAgent: physical disks ensure: Updated storage monitor file = sled-agent/src/sled_agent.rs:921 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
4502024-11-02T03:55:23.806ZINFOSledAgent: physical disks ensure: Updated zone bundler file = sled-agent/src/sled_agent.rs:926 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
4512024-11-02T03:55:23.806ZINFOSledAgent (ProbeManager): use_only_these_disks: Processing new request file = sled-agent/src/probe_manager.rs:122 gen = Generation(1) sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
4522024-11-02T03:55:23.806ZINFOSledAgent: physical disks ensure: Updated probes file = sled-agent/src/sled_agent.rs:932 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
4532024-11-02T03:55:23.806ZINFOSledAgent (InstanceManager): use_only_these_disks: Processing new request file = sled-agent/src/instance_manager.rs:799 gen = Generation(1)
4542024-11-02T03:55:23.806ZINFOSledAgent: physical disks ensure: Updated instances file = sled-agent/src/sled_agent.rs:937 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
4552024-11-02T03:55:23.806ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 3728526 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:64176 req_id = 2edd3ab7-b3a4-4a68-bcd3-fc2a8aec563b response_code = 200 uri = /omicron-physical-disks
4562024-11-02T03:55:23.837ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(2), zones: [OmicronZoneConfig { id: 2c96eacb-1841-4277-9578-6b84caeef9c6 (service), filesystem_pool: Some(ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 1a7f019f-e208-4fae-b98e-fe6120f9b5e2 (service), filesystem_pool: Some(ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 0c3f42f2-ce05-428f-a91a-f78ae70ccccf (service), filesystem_pool: Some(ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }] } file = sled-agent/src/rack_setup/service.rs:459 sled_address = [fd00:1122:3344:101::1]:12345
4572024-11-02T03:55:23.837ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:46159
4582024-11-02T03:55:23.837ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2c96eacb-1841-4277-9578-6b84caeef9c6, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
4592024-11-02T03:55:23.837ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 2c96eacb-1841-4277-9578-6b84caeef9c6, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: InternalDns }, responder: ... }
4602024-11-02T03:55:23.888ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 1a7f019f-e208-4fae-b98e-fe6120f9b5e2, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
4612024-11-02T03:55:23.888ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 1a7f019f-e208-4fae-b98e-fe6120f9b5e2, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: InternalDns }, responder: ... }
4622024-11-02T03:55:23.937ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 0c3f42f2-ce05-428f-a91a-f78ae70ccccf, dataset_name: DatasetName { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
4632024-11-02T03:55:23.938ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 0c3f42f2-ce05-428f-a91a-f78ae70ccccf, dataset_name: DatasetName { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }, kind: InternalDns }, responder: ... }
4642024-11-02T03:55:23.987ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
4652024-11-02T03:55:23.987ZINFOSledAgent (ServiceManager): No ledger in /pool/int/6308f1ec-c08e-4282-a1c2-db37d6b8a044/config/omicron-zones.json file = common/src/ledger.rs:188
4662024-11-02T03:55:23.987ZINFOSledAgent (ServiceManager): No ledger in /pool/int/fda73319-9c6d-44d8-9377-b69537374c59/config/omicron-zones.json file = common/src/ledger.rs:188
4672024-11-02T03:55:23.987ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
4682024-11-02T03:55:24.031ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
4692024-11-02T03:55:24.031ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
4702024-11-02T03:55:24.031ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
4712024-11-02T03:55:24.082ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_2c96eacb-1841-4277-9578-6b84caeef9c6 file = illumos-utils/src/zone.rs:325
4722024-11-02T03:55:24.086ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_0c3f42f2-ce05-428f-a91a-f78ae70ccccf file = illumos-utils/src/zone.rs:325
4732024-11-02T03:55:24.089ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_1a7f019f-e208-4fae-b98e-fe6120f9b5e2 file = illumos-utils/src/zone.rs:325
4742024-11-02T03:55:24.102ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_2c96eacb-1841-4277-9578-6b84caeef9c6 file = illumos-utils/src/zone.rs:363
4752024-11-02T03:55:24.106ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_0c3f42f2-ce05-428f-a91a-f78ae70ccccf file = illumos-utils/src/zone.rs:363
4762024-11-02T03:55:24.110ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_1a7f019f-e208-4fae-b98e-fe6120f9b5e2 file = illumos-utils/src/zone.rs:363
4772024-11-02T03:55:24.653ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/5b8cdd5a-f021-45f2-8323-b6c8f9776060) file = sled-agent/src/probe_manager.rs:231 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
4782024-11-02T03:55:25.622ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_0c3f42f2-ce05-428f-a91a-f78ae70ccccf: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService2'/> <propval type="astring" name="gateway" value='fd00:1122:3344:3::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:3::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:3::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:3::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
4792024-11-02T03:55:25.623ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:3::, width: 64 }
4802024-11-02T03:55:25.623ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_0c3f42f2-ce05-428f-a91a-f78ae70ccccf zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_0c3f42f2-ce05-428f-a91a-f78ae70ccccf
4812024-11-02T03:55:26.105ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_1a7f019f-e208-4fae-b98e-fe6120f9b5e2: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService3'/> <propval type="astring" name="gateway" value='fd00:1122:3344:2::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:2::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:2::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:2::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
4822024-11-02T03:55:26.105ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:2::, width: 64 }
4832024-11-02T03:55:26.587ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_2c96eacb-1841-4277-9578-6b84caeef9c6: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService1'/> <propval type="astring" name="gateway" value='fd00:1122:3344:1::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:1::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:1::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:1::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
4842024-11-02T03:55:26.594ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_0c3f42f2-ce05-428f-a91a-f78ae70ccccf") failed: Property not found. retry in 62.553583ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_0c3f42f2-ce05-428f-a91a-f78ae70ccccf
4852024-11-02T03:55:26.594ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_1a7f019f-e208-4fae-b98e-fe6120f9b5e2 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_1a7f019f-e208-4fae-b98e-fe6120f9b5e2
4862024-11-02T03:55:26.597ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:1::, width: 64 }
4872024-11-02T03:55:26.597ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_2c96eacb-1841-4277-9578-6b84caeef9c6 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_2c96eacb-1841-4277-9578-6b84caeef9c6
4882024-11-02T03:55:26.666ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_0c3f42f2-ce05-428f-a91a-f78ae70ccccf") failed: Property not found. retry in 94.955396ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_0c3f42f2-ce05-428f-a91a-f78ae70ccccf
4892024-11-02T03:55:26.716ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_2c96eacb-1841-4277-9578-6b84caeef9c6") failed: Property not found. retry in 47.65661ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_2c96eacb-1841-4277-9578-6b84caeef9c6
4902024-11-02T03:55:26.723ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_1a7f019f-e208-4fae-b98e-fe6120f9b5e2") failed: Property not found. retry in 47.997819ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_1a7f019f-e208-4fae-b98e-fe6120f9b5e2
4912024-11-02T03:55:26.769ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_0c3f42f2-ce05-428f-a91a-f78ae70ccccf") failed: Property not found. retry in 226.246328ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_0c3f42f2-ce05-428f-a91a-f78ae70ccccf
4922024-11-02T03:55:26.775ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_2c96eacb-1841-4277-9578-6b84caeef9c6") failed: Property not found. retry in 135.7428ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_2c96eacb-1841-4277-9578-6b84caeef9c6
4932024-11-02T03:55:26.789ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_1a7f019f-e208-4fae-b98e-fe6120f9b5e2") failed: Property not found. retry in 148.702223ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_1a7f019f-e208-4fae-b98e-fe6120f9b5e2
4942024-11-02T03:55:26.941ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_2c96eacb-1841-4277-9578-6b84caeef9c6") failed: Property not found. retry in 290.450482ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_2c96eacb-1841-4277-9578-6b84caeef9c6
4952024-11-02T03:55:26.957ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_1a7f019f-e208-4fae-b98e-fe6120f9b5e2") failed: Property not found. retry in 164.109803ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_1a7f019f-e208-4fae-b98e-fe6120f9b5e2
4962024-11-02T03:55:27.030ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_0c3f42f2-ce05-428f-a91a-f78ae70ccccf") failed: Property not found. retry in 566.968512ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_0c3f42f2-ce05-428f-a91a-f78ae70ccccf
4972024-11-02T03:55:27.147ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_1a7f019f-e208-4fae-b98e-fe6120f9b5e2") failed: Property not found. retry in 328.48286ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_1a7f019f-e208-4fae-b98e-fe6120f9b5e2
4982024-11-02T03:55:27.242ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_2c96eacb-1841-4277-9578-6b84caeef9c6") failed: Property not found. retry in 266.647007ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_2c96eacb-1841-4277-9578-6b84caeef9c6
4992024-11-02T03:55:27.484ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_1a7f019f-e208-4fae-b98e-fe6120f9b5e2") failed: Property not found. retry in 758.060303ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_1a7f019f-e208-4fae-b98e-fe6120f9b5e2
5002024-11-02T03:55:27.518ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_2c96eacb-1841-4277-9578-6b84caeef9c6") failed: Property not found. retry in 1.185677121s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_2c96eacb-1841-4277-9578-6b84caeef9c6
5012024-11-02T03:55:27.605ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_0c3f42f2-ce05-428f-a91a-f78ae70ccccf") failed: Property not found. retry in 601.500775ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_0c3f42f2-ce05-428f-a91a-f78ae70ccccf
5022024-11-02T03:55:28.217ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_0c3f42f2-ce05-428f-a91a-f78ae70ccccf") failed: Property not found. retry in 1.37697285s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_0c3f42f2-ce05-428f-a91a-f78ae70ccccf
5032024-11-02T03:55:28.252ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_1a7f019f-e208-4fae-b98e-fe6120f9b5e2") failed: Property not found. retry in 595.975988ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_1a7f019f-e208-4fae-b98e-fe6120f9b5e2
5042024-11-02T03:55:28.712ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_2c96eacb-1841-4277-9578-6b84caeef9c6") failed: Property not found. retry in 1.35765458s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_2c96eacb-1841-4277-9578-6b84caeef9c6
5052024-11-02T03:55:28.857ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_1a7f019f-e208-4fae-b98e-fe6120f9b5e2") failed: Property not found. retry in 716.968114ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_1a7f019f-e208-4fae-b98e-fe6120f9b5e2
5062024-11-02T03:55:29.238ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::2 exists - OK file = sled-agent/src/services.rs:4254
5072024-11-02T03:55:29.238ZINFOSledAgent (ServiceManager): Ensuring there is a default route file = sled-agent/src/services.rs:4264 gateway = fd00:1122:3344:101::1
5082024-11-02T03:55:29.244ZINFOSledAgent (ServiceManager): configuring dendrite service file = sled-agent/src/services.rs:4343
5092024-11-02T03:55:29.346ZINFOSledAgent (ServiceManager): refreshed dendrite service with new configuration file = sled-agent/src/services.rs:4391
5102024-11-02T03:55:29.346ZINFOSledAgent (ServiceManager): configuring lldp service file = sled-agent/src/services.rs:4418
5112024-11-02T03:55:29.385ZINFOSledAgent (ServiceManager): refreshed lldpd service with new configuration file = sled-agent/src/services.rs:4431
5122024-11-02T03:55:29.385ZINFOSledAgent (ServiceManager): configuring MGS service file = sled-agent/src/services.rs:4295
5132024-11-02T03:55:29.435ZINFOSledAgent (ServiceManager): refreshed MGS service with new configuration file = sled-agent/src/services.rs:4337
5142024-11-02T03:55:29.435ZINFOSledAgent (ServiceManager): configuring wicketd file = sled-agent/src/services.rs:4398 rack_subnet = fd00:1122:3344::
5152024-11-02T03:55:29.453ZINFOSledAgent (ServiceManager): refreshed wicketd service with new configuration file = sled-agent/src/services.rs:4409
5162024-11-02T03:55:29.453ZINFOSledAgent (ServiceManager): configuring mgd service file = sled-agent/src/services.rs:4451
5172024-11-02T03:55:29.518ZINFOSledAgent (ServiceManager): refreshed mgd service with new configuration file = sled-agent/src/services.rs:4483
5182024-11-02T03:55:29.518ZINFOSledAgent (ServiceManager): configuring mg-ddm service file = sled-agent/src/services.rs:4489
5192024-11-02T03:55:29.522ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 12924275829650550807 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
5202024-11-02T03:55:29.523ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 11751738358168414211 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
5212024-11-02T03:55:29.583ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_1a7f019f-e208-4fae-b98e-fe6120f9b5e2") failed: Property not found. retry in 860.282166ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_1a7f019f-e208-4fae-b98e-fe6120f9b5e2
5222024-11-02T03:55:29.603ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_0c3f42f2-ce05-428f-a91a-f78ae70ccccf") failed: Property not found. retry in 1.003374207s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_0c3f42f2-ce05-428f-a91a-f78ae70ccccf
5232024-11-02T03:55:29.608ZINFOSledAgent (ServiceManager): refreshed mg-ddm service with new configuration file = sled-agent/src/services.rs:4530
5242024-11-02T03:55:29.608ZINFOSledAgent (ServiceManager): Determining physical location of our switch zone at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:370
5252024-11-02T03:55:29.641ZWARNSledAgent (ServiceManager): Failed to get switch ID from MGS (retrying in 73.450423ms) error = Communication Error: error sending request for url (http://[fd00:1122:3344:101::2]:12225/local/switch-id) file = sled-agent/src/bootstrap/early_networking.rs:389
5262024-11-02T03:55:29.715ZWARNSledAgent (ServiceManager): Failed to get switch ID from MGS (retrying in 123.789035ms) error = Communication Error: error sending request for url (http://[fd00:1122:3344:101::2]:12225/local/switch-id) file = sled-agent/src/bootstrap/early_networking.rs:389
5272024-11-02T03:55:29.840ZWARNSledAgent (ServiceManager): Failed to get switch ID from MGS (retrying in 178.861297ms) error = Communication Error: error sending request for url (http://[fd00:1122:3344:101::2]:12225/local/switch-id) file = sled-agent/src/bootstrap/early_networking.rs:389
5282024-11-02T03:55:29.999ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::2]:57432
5292024-11-02T03:55:29.999ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 60 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::2]:57432 req_id = b0b4db40-b3b0-491c-8aad-969295fd644e response_code = 200 uri = /sled-identifiers
5302024-11-02T03:55:30.021ZINFOSledAgent (ServiceManager): Initializing 1 Uplinks on Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:418
5312024-11-02T03:55:30.053ZINFOSledAgent (ServiceManager): Building Port Configuration file = sled-agent/src/bootstrap/early_networking.rs:686
5322024-11-02T03:55:30.053ZINFOSledAgent (ServiceManager): Checking dendrite uptime file = sled-agent/src/bootstrap/early_networking.rs:731
5332024-11-02T03:55:30.053ZINFOSledAgent (ServiceManager): Dendrite online file = sled-agent/src/bootstrap/early_networking.rs:734 uptime = 1
5342024-11-02T03:55:30.053ZINFOSledAgent (ServiceManager): Configuring default uplink on switch config = PortSettings {\n links: {\n "0": LinkSettings {\n addrs: [\n 10.151.1.174,\n ],\n params: LinkCreate {\n autoneg: false,\n fec: None,\n kr: false,\n lane: Some(\n LinkId(\n 0,\n ),\n ),\n speed: Speed40G,\n tx_eq: None,\n },\n },\n },\n} file = sled-agent/src/bootstrap/early_networking.rs:440
5352024-11-02T03:55:30.094ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_2c96eacb-1841-4277-9578-6b84caeef9c6") failed: Property not found. retry in 631.002278ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_2c96eacb-1841-4277-9578-6b84caeef9c6
5362024-11-02T03:55:30.096ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
5372024-11-02T03:55:30.146ZINFOSledAgent (StorageManager): Received DetectedRawDisksChanged { raw_disks: {}, tx: ... } file = sled-storage/src/manager.rs:451
5382024-11-02T03:55:30.146ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
5392024-11-02T03:55:30.146ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
5402024-11-02T03:55:30.146ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
5412024-11-02T03:55:30.146ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
5422024-11-02T03:55:30.146ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
5432024-11-02T03:55:30.146ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
5442024-11-02T03:55:30.146ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
5452024-11-02T03:55:30.146ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
5462024-11-02T03:55:30.146ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
5472024-11-02T03:55:30.146ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
5482024-11-02T03:55:30.146ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
5492024-11-02T03:55:30.146ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
5502024-11-02T03:55:30.146ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
5512024-11-02T03:55:30.146ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
5522024-11-02T03:55:30.146ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
5532024-11-02T03:55:30.146ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
5542024-11-02T03:55:30.147ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
5552024-11-02T03:55:30.147ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
5562024-11-02T03:55:30.147ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
5572024-11-02T03:55:30.147ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
5582024-11-02T03:55:30.147ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
5592024-11-02T03:55:30.147ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
5602024-11-02T03:55:30.453ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_1a7f019f-e208-4fae-b98e-fe6120f9b5e2") failed: Property not found. retry in 533.033832ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_1a7f019f-e208-4fae-b98e-fe6120f9b5e2
5612024-11-02T03:55:30.614ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_0c3f42f2-ce05-428f-a91a-f78ae70ccccf") failed: Property not found. retry in 861.417872ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_0c3f42f2-ce05-428f-a91a-f78ae70ccccf
5622024-11-02T03:55:30.640ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 3855537906354135706 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
5632024-11-02T03:55:30.640ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 11546169058853415843 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
5642024-11-02T03:55:30.734ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_2c96eacb-1841-4277-9578-6b84caeef9c6") failed: Property not found. retry in 510.754397ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_2c96eacb-1841-4277-9578-6b84caeef9c6
5652024-11-02T03:55:30.996ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_1a7f019f-e208-4fae-b98e-fe6120f9b5e2") failed: Property not found. retry in 904.365323ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_1a7f019f-e208-4fae-b98e-fe6120f9b5e2
5662024-11-02T03:55:31.255ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_2c96eacb-1841-4277-9578-6b84caeef9c6") failed: Property not found. retry in 1.098716234s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_2c96eacb-1841-4277-9578-6b84caeef9c6
5672024-11-02T03:55:32.372ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3273 zone = oxz_internal_dns_2c96eacb-1841-4277-9578-6b84caeef9c6
5682024-11-02T03:55:32.372ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3273 zone = oxz_internal_dns_0c3f42f2-ce05-428f-a91a-f78ae70ccccf
5692024-11-02T03:55:32.372ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3273 zone = oxz_internal_dns_1a7f019f-e208-4fae-b98e-fe6120f9b5e2
5702024-11-02T03:55:32.372ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/6308f1ec-c08e-4282-a1c2-db37d6b8a044/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
5712024-11-02T03:55:32.373ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/fda73319-9c6d-44d8-9377-b69537374c59/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
5722024-11-02T03:55:32.373ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 8535845 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:46159 req_id = 323c4f85-8353-40c7-98d2-1c3c62889df9 response_code = 204 uri = /omicron-zones
5732024-11-02T03:55:32.375ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:1::1]:5353 file = sled-agent/src/rack_setup/service.rs:576
5742024-11-02T03:55:32.422ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:2::1]:5353 file = sled-agent/src/rack_setup/service.rs:576
5752024-11-02T03:55:32.466ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:3::1]:5353 file = sled-agent/src/rack_setup/service.rs:576
5762024-11-02T03:55:32.518ZINFOSledAgent (RSS): Configured all DNS servers file = sled-agent/src/rack_setup/service.rs:612
5772024-11-02T03:55:32.518ZINFOSledAgent (RSS): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
5782024-11-02T03:55:32.550ZINFOSledAgent (RSS): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
5792024-11-02T03:55:32.551ZINFOSledAgent (RSS): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
5802024-11-02T03:55:32.551ZINFOSledAgent (RSS): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
5812024-11-02T03:55:32.579ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(3), zones: [OmicronZoneConfig { id: 2c96eacb-1841-4277-9578-6b84caeef9c6 (service), filesystem_pool: Some(ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 1a7f019f-e208-4fae-b98e-fe6120f9b5e2 (service), filesystem_pool: Some(ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 0c3f42f2-ce05-428f-a91a-f78ae70ccccf (service), filesystem_pool: Some(ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 2d705ba5-6076-4a40-8774-81749916100b (service), filesystem_pool: Some(ZpoolName { id: 010339bd-18fb-4f1c-9455-7caf9e67e152 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 80c1df58-d201-4881-943a-0c6ca851a8d9, kind: Service { id: 2d705ba5-6076-4a40-8774-81749916100b }, name: Name("ntp-2d705ba5-6076-4a40-8774-81749916100b"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 255, 238])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } } }] } file = sled-agent/src/rack_setup/service.rs:459 sled_address = [fd00:1122:3344:101::1]:12345
5822024-11-02T03:55:32.580ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:62149
5832024-11-02T03:55:32.580ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2c96eacb-1841-4277-9578-6b84caeef9c6, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
5842024-11-02T03:55:32.580ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 2c96eacb-1841-4277-9578-6b84caeef9c6, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: InternalDns }, responder: ... }
5852024-11-02T03:55:32.601ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 1a7f019f-e208-4fae-b98e-fe6120f9b5e2, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
5862024-11-02T03:55:32.601ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 1a7f019f-e208-4fae-b98e-fe6120f9b5e2, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: InternalDns }, responder: ... }
5872024-11-02T03:55:32.619ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 0c3f42f2-ce05-428f-a91a-f78ae70ccccf, dataset_name: DatasetName { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
5882024-11-02T03:55:32.619ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 0c3f42f2-ce05-428f-a91a-f78ae70ccccf, dataset_name: DatasetName { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }, kind: InternalDns }, responder: ... }
5892024-11-02T03:55:32.637ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
5902024-11-02T03:55:32.638ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/6308f1ec-c08e-4282-a1c2-db37d6b8a044/config/omicron-zones.json file = common/src/ledger.rs:177
5912024-11-02T03:55:32.638ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/fda73319-9c6d-44d8-9377-b69537374c59/config/omicron-zones.json file = common/src/ledger.rs:177
5922024-11-02T03:55:32.638ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
5932024-11-02T03:55:32.648ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
5942024-11-02T03:55:32.680ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
5952024-11-02T03:55:32.681ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
5962024-11-02T03:55:32.681ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
5972024-11-02T03:55:32.710ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:430 port = Port { inner: PortInner(PortData { name: "opte0", ip: 172.30.3.5, mac: MacAddr6([168, 64, 37, 255, 255, 238]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.3.1 } }) }
5982024-11-02T03:55:32.710ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1260 zone_type = boundary_ntp
5992024-11-02T03:55:32.710ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.1.169 file = clients/dpd-client/src/lib.rs:83
6002024-11-02T03:55:32.711ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.1.169 file = clients/dpd-client/src/lib.rs:90
6012024-11-02T03:55:32.711ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.1.169 file = clients/dpd-client/src/lib.rs:118
6022024-11-02T03:55:32.711ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
6032024-11-02T03:55:32.734ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_ntp_2d705ba5-6076-4a40-8774-81749916100b file = illumos-utils/src/zone.rs:325
6042024-11-02T03:55:32.752ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_ntp_2d705ba5-6076-4a40-8774-81749916100b file = illumos-utils/src/zone.rs:363
6052024-11-02T03:55:33.370ZINFOSledAgent (ServiceManager): Profile for oxz_ntp_2d705ba5-6076-4a40-8774-81749916100b: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService4'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::1b'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/chrony-setup"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="allow" value='fd00:1122:3344:100::/56'/> <propval type="boolean" name="boundary" value='true'/> <propval type="astring" name="boundary_pool" value='boundary-ntp.control-plane.oxide.internal'/> <propval type="astring" name="server" value='0.pool.ntp.org'/> </property_group> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="1.1.1.1" /> <value_node value="9.9.9.9" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/ntp"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte0'/> <propval type="astring" name="gateway" value='172.30.3.1'/> <propval type="astring" name="ip" value='172.30.3.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
6062024-11-02T03:55:33.370ZINFOSledAgent (ServiceManager): Booting oxz_ntp_2d705ba5-6076-4a40-8774-81749916100b zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_ntp_2d705ba5-6076-4a40-8774-81749916100b
6072024-11-02T03:55:33.509ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_2d705ba5-6076-4a40-8774-81749916100b") failed: Property not found. retry in 50.74329ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_2d705ba5-6076-4a40-8774-81749916100b
6082024-11-02T03:55:33.569ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_2d705ba5-6076-4a40-8774-81749916100b") failed: Property not found. retry in 103.647235ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_2d705ba5-6076-4a40-8774-81749916100b
6092024-11-02T03:55:33.705ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_2d705ba5-6076-4a40-8774-81749916100b") failed: Property not found. retry in 270.410963ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_2d705ba5-6076-4a40-8774-81749916100b
6102024-11-02T03:55:33.986ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_2d705ba5-6076-4a40-8774-81749916100b") failed: Property not found. retry in 426.244485ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_2d705ba5-6076-4a40-8774-81749916100b
6112024-11-02T03:55:34.421ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_2d705ba5-6076-4a40-8774-81749916100b") failed: Property not found. retry in 817.763273ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_2d705ba5-6076-4a40-8774-81749916100b
6122024-11-02T03:55:35.248ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_2d705ba5-6076-4a40-8774-81749916100b") failed: Property not found. retry in 902.771058ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_2d705ba5-6076-4a40-8774-81749916100b
6132024-11-02T03:55:36.160ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_2d705ba5-6076-4a40-8774-81749916100b") failed: Property not found. retry in 1.269222291s file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_2d705ba5-6076-4a40-8774-81749916100b
6142024-11-02T03:55:37.439ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_2d705ba5-6076-4a40-8774-81749916100b") failed: Property not found. retry in 524.322523ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_2d705ba5-6076-4a40-8774-81749916100b
6152024-11-02T03:55:37.974ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_2d705ba5-6076-4a40-8774-81749916100b") failed: Property not found. retry in 1.371212835s file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_2d705ba5-6076-4a40-8774-81749916100b
6162024-11-02T03:55:39.365ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3273 zone = oxz_ntp_2d705ba5-6076-4a40-8774-81749916100b
6172024-11-02T03:55:39.365ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/6308f1ec-c08e-4282-a1c2-db37d6b8a044/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
6182024-11-02T03:55:39.365ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/fda73319-9c6d-44d8-9377-b69537374c59/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
6192024-11-02T03:55:39.366ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 6785996 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:62149 req_id = 137df505-3a01-4632-942f-4c152a013b20 response_code = 204 uri = /omicron-zones
6202024-11-02T03:55:39.368ZINFOSledAgent (RSS): Waiting for rack time synchronization file = sled-agent/src/rack_setup/service.rs:653
6212024-11-02T03:55:39.397ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:633
6222024-11-02T03:55:39.397ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:62803
6232024-11-02T03:55:39.406ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_2d705ba5-6076-4a40-8774-81749916100b': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3862
6242024-11-02T03:55:39.406ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:938 latency_us = 8714 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:62803 req_id = 1c1d9613-0413-4674-a74d-aa8c5cc68067 response_code = 500 uri = /timesync
6252024-11-02T03:55:39.408ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:683
6262024-11-02T03:55:39.525ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 12924275829650550807 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
6272024-11-02T03:55:39.525ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 11751738358168414211 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
6282024-11-02T03:55:39.577ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:633
6292024-11-02T03:55:39.577ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:52195
6302024-11-02T03:55:40.642ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 3855537906354135706 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
6312024-11-02T03:55:40.642ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 11546169058853415843 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
6322024-11-02T03:55:41.499ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 2321050458409999222 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
6332024-11-02T03:55:41.924ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 9692351653165157743 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
6342024-11-02T03:55:42.375ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 1963514396745904249 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
6352024-11-02T03:55:43.586ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_2d705ba5-6076-4a40-8774-81749916100b': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3862
6362024-11-02T03:55:43.586ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:938 latency_us = 4008976 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:52195 req_id = f41f4f32-8bd0-475b-881b-69988ff0e120 response_code = 500 uri = /timesync
6372024-11-02T03:55:43.589ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:683
6382024-11-02T03:55:43.732ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:633
6392024-11-02T03:55:43.732ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:56328
6402024-11-02T03:55:43.740ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 8278 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:56328 req_id = ea1d78ae-d62b-4374-9a3e-c964db85b6db response_code = 200 uri = /timesync
6412024-11-02T03:55:43.743ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:661
6422024-11-02T03:55:43.743ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:683
6432024-11-02T03:55:43.858ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:633
6442024-11-02T03:55:43.858ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:50785
6452024-11-02T03:55:43.867ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 8515 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:50785 req_id = b9b3c107-6273-480d-97b4-c31272577752 response_code = 200 uri = /timesync
6462024-11-02T03:55:43.870ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:661
6472024-11-02T03:55:43.870ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:683
6482024-11-02T03:55:44.148ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:633
6492024-11-02T03:55:44.148ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:57861
6502024-11-02T03:55:44.155ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 7275 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:57861 req_id = 583d65dc-18e6-4f8a-8330-f3633dfd9a13 response_code = 200 uri = /timesync
6512024-11-02T03:55:44.158ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:661
6522024-11-02T03:55:44.158ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:683
6532024-11-02T03:55:44.483ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:633
6542024-11-02T03:55:44.483ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:62504
6552024-11-02T03:55:44.491ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 8347 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:62504 req_id = c7c976be-8611-4252-b054-82885ad05244 response_code = 200 uri = /timesync
6562024-11-02T03:55:44.494ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:661
6572024-11-02T03:55:44.494ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:683
6582024-11-02T03:55:44.695ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:633
6592024-11-02T03:55:44.695ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:58533
6602024-11-02T03:55:44.704ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 8085 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:58533 req_id = 79688de4-e170-4593-8f54-14d82de10458 response_code = 200 uri = /timesync
6612024-11-02T03:55:44.705ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:661
6622024-11-02T03:55:44.705ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:683
6632024-11-02T03:55:45.157ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:633
6642024-11-02T03:55:45.157ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:50706
6652024-11-02T03:55:45.165ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 7624 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:50706 req_id = 875458d1-a97e-4420-b20a-dc827befe095 response_code = 200 uri = /timesync
6662024-11-02T03:55:45.168ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.042182788 } file = sled-agent/src/rack_setup/service.rs:661
6672024-11-02T03:55:45.168ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:683
6682024-11-02T03:55:45.681ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:633
6692024-11-02T03:55:45.681ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:46207
6702024-11-02T03:55:45.702ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 20770 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:46207 req_id = 49f709e0-546e-499d-9542-1345239bb47f response_code = 200 uri = /timesync
6712024-11-02T03:55:45.705ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: true, ref_id: 394502151, ip_addr: 23.131.160.7, stratum: 3, ref_time: 1730519745.3583913, correction: 0.041070238 } file = sled-agent/src/rack_setup/service.rs:661
6722024-11-02T03:55:45.705ZINFOSledAgent (RSS): Finished setting up Internal DNS and NTP file = sled-agent/src/rack_setup/service.rs:1308
6732024-11-02T03:55:45.734ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 2c96eacb-1841-4277-9578-6b84caeef9c6 (service), filesystem_pool: Some(ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 1a7f019f-e208-4fae-b98e-fe6120f9b5e2 (service), filesystem_pool: Some(ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 0c3f42f2-ce05-428f-a91a-f78ae70ccccf (service), filesystem_pool: Some(ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 2d705ba5-6076-4a40-8774-81749916100b (service), filesystem_pool: Some(ZpoolName { id: 010339bd-18fb-4f1c-9455-7caf9e67e152 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 80c1df58-d201-4881-943a-0c6ca851a8d9, kind: Service { id: 2d705ba5-6076-4a40-8774-81749916100b }, name: Name("ntp-2d705ba5-6076-4a40-8774-81749916100b"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 255, 238])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: ea8c4184-9256-4357-bb1c-db39d54e6a2e (service), filesystem_pool: Some(ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External } } } }, OmicronZoneConfig { id: 41710ebd-a7df-4192-87ee-821f3687cee3 (service), filesystem_pool: Some(ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External } } } }, OmicronZoneConfig { id: 5b85d391-0238-40a9-b228-ddf2fb022360 (service), filesystem_pool: Some(ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External } } } }, OmicronZoneConfig { id: 4479233d-49d6-4267-9076-4d640d54f1e2 (service), filesystem_pool: Some(ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External } } } }, OmicronZoneConfig { id: 37b81eec-9fbb-4376-aec7-36daed3465bc (service), filesystem_pool: Some(ZpoolName { id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:459 sled_address = [fd00:1122:3344:101::1]:12345
6742024-11-02T03:55:45.735ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:59094
6752024-11-02T03:55:45.735ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2c96eacb-1841-4277-9578-6b84caeef9c6, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
6762024-11-02T03:55:45.735ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 2c96eacb-1841-4277-9578-6b84caeef9c6, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: InternalDns }, responder: ... }
6772024-11-02T03:55:45.756ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 1a7f019f-e208-4fae-b98e-fe6120f9b5e2, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
6782024-11-02T03:55:45.757ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 1a7f019f-e208-4fae-b98e-fe6120f9b5e2, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: InternalDns }, responder: ... }
6792024-11-02T03:55:45.776ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 0c3f42f2-ce05-428f-a91a-f78ae70ccccf, dataset_name: DatasetName { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
6802024-11-02T03:55:45.776ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 0c3f42f2-ce05-428f-a91a-f78ae70ccccf, dataset_name: DatasetName { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }, kind: InternalDns }, responder: ... }
6812024-11-02T03:55:45.794ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ea8c4184-9256-4357-bb1c-db39d54e6a2e, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
6822024-11-02T03:55:45.794ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: ea8c4184-9256-4357-bb1c-db39d54e6a2e, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: Cockroach }, responder: ... }
6832024-11-02T03:55:45.844ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 41710ebd-a7df-4192-87ee-821f3687cee3, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
6842024-11-02T03:55:45.844ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 41710ebd-a7df-4192-87ee-821f3687cee3, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: Cockroach }, responder: ... }
6852024-11-02T03:55:45.896ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 5b85d391-0238-40a9-b228-ddf2fb022360, dataset_name: DatasetName { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
6862024-11-02T03:55:45.896ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 5b85d391-0238-40a9-b228-ddf2fb022360, dataset_name: DatasetName { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }, kind: Cockroach }, responder: ... }
6872024-11-02T03:55:45.947ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 4479233d-49d6-4267-9076-4d640d54f1e2, dataset_name: DatasetName { pool_name: ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
6882024-11-02T03:55:45.947ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 4479233d-49d6-4267-9076-4d640d54f1e2, dataset_name: DatasetName { pool_name: ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External }, kind: Cockroach }, responder: ... }
6892024-11-02T03:55:45.998ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 37b81eec-9fbb-4376-aec7-36daed3465bc, dataset_name: DatasetName { pool_name: ZpoolName { id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
6902024-11-02T03:55:45.998ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 37b81eec-9fbb-4376-aec7-36daed3465bc, dataset_name: DatasetName { pool_name: ZpoolName { id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool), kind: External }, kind: Cockroach }, responder: ... }
6912024-11-02T03:55:46.052ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
6922024-11-02T03:55:46.052ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/6308f1ec-c08e-4282-a1c2-db37d6b8a044/config/omicron-zones.json file = common/src/ledger.rs:177
6932024-11-02T03:55:46.052ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/fda73319-9c6d-44d8-9377-b69537374c59/config/omicron-zones.json file = common/src/ledger.rs:177
6942024-11-02T03:55:46.052ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
6952024-11-02T03:55:46.085ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3277 zone = oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2
6962024-11-02T03:55:46.085ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3277 zone = oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc
6972024-11-02T03:55:46.085ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3277 zone = oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e
6982024-11-02T03:55:46.085ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3277 zone = oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360
6992024-11-02T03:55:46.085ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3277 zone = oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3
7002024-11-02T03:55:46.085ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2", TimeNotSynchronized), ("oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc", TimeNotSynchronized), ("oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e", TimeNotSynchronized), ("oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360", TimeNotSynchronized), ("oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:938 latency_us = 350313 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:59094 req_id = 1682fe08-d0f9-44ea-8214-a3779f09671b response_code = 500 uri = /omicron-zones
7012024-11-02T03:55:46.085ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "1682fe08-d0f9-44ea-8214-a3779f09671b", "content-length": "124", "date": "Sat, 02 Nov 2024 03:55:45 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "1682fe08-d0f9-44ea-8214-a3779f09671b" } file = sled-agent/src/rack_setup/service.rs:496 retry_after = 94.695105ms sled_address = [fd00:1122:3344:101::1]:12345
7022024-11-02T03:55:46.182ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 2c96eacb-1841-4277-9578-6b84caeef9c6 (service), filesystem_pool: Some(ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 1a7f019f-e208-4fae-b98e-fe6120f9b5e2 (service), filesystem_pool: Some(ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 0c3f42f2-ce05-428f-a91a-f78ae70ccccf (service), filesystem_pool: Some(ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 2d705ba5-6076-4a40-8774-81749916100b (service), filesystem_pool: Some(ZpoolName { id: 010339bd-18fb-4f1c-9455-7caf9e67e152 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 80c1df58-d201-4881-943a-0c6ca851a8d9, kind: Service { id: 2d705ba5-6076-4a40-8774-81749916100b }, name: Name("ntp-2d705ba5-6076-4a40-8774-81749916100b"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 255, 238])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: ea8c4184-9256-4357-bb1c-db39d54e6a2e (service), filesystem_pool: Some(ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External } } } }, OmicronZoneConfig { id: 41710ebd-a7df-4192-87ee-821f3687cee3 (service), filesystem_pool: Some(ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External } } } }, OmicronZoneConfig { id: 5b85d391-0238-40a9-b228-ddf2fb022360 (service), filesystem_pool: Some(ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External } } } }, OmicronZoneConfig { id: 4479233d-49d6-4267-9076-4d640d54f1e2 (service), filesystem_pool: Some(ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External } } } }, OmicronZoneConfig { id: 37b81eec-9fbb-4376-aec7-36daed3465bc (service), filesystem_pool: Some(ZpoolName { id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:459 sled_address = [fd00:1122:3344:101::1]:12345
7032024-11-02T03:55:46.182ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2c96eacb-1841-4277-9578-6b84caeef9c6, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
7042024-11-02T03:55:46.182ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 2c96eacb-1841-4277-9578-6b84caeef9c6, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: InternalDns }, responder: ... }
7052024-11-02T03:55:46.201ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 1a7f019f-e208-4fae-b98e-fe6120f9b5e2, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
7062024-11-02T03:55:46.201ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 1a7f019f-e208-4fae-b98e-fe6120f9b5e2, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: InternalDns }, responder: ... }
7072024-11-02T03:55:46.219ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 0c3f42f2-ce05-428f-a91a-f78ae70ccccf, dataset_name: DatasetName { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
7082024-11-02T03:55:46.219ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 0c3f42f2-ce05-428f-a91a-f78ae70ccccf, dataset_name: DatasetName { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }, kind: InternalDns }, responder: ... }
7092024-11-02T03:55:46.237ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ea8c4184-9256-4357-bb1c-db39d54e6a2e, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
7102024-11-02T03:55:46.237ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: ea8c4184-9256-4357-bb1c-db39d54e6a2e, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: Cockroach }, responder: ... }
7112024-11-02T03:55:46.255ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 41710ebd-a7df-4192-87ee-821f3687cee3, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
7122024-11-02T03:55:46.255ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 41710ebd-a7df-4192-87ee-821f3687cee3, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: Cockroach }, responder: ... }
7132024-11-02T03:55:46.272ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 5b85d391-0238-40a9-b228-ddf2fb022360, dataset_name: DatasetName { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
7142024-11-02T03:55:46.272ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 5b85d391-0238-40a9-b228-ddf2fb022360, dataset_name: DatasetName { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }, kind: Cockroach }, responder: ... }
7152024-11-02T03:55:46.290ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 4479233d-49d6-4267-9076-4d640d54f1e2, dataset_name: DatasetName { pool_name: ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
7162024-11-02T03:55:46.290ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 4479233d-49d6-4267-9076-4d640d54f1e2, dataset_name: DatasetName { pool_name: ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External }, kind: Cockroach }, responder: ... }
7172024-11-02T03:55:46.307ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 37b81eec-9fbb-4376-aec7-36daed3465bc, dataset_name: DatasetName { pool_name: ZpoolName { id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
7182024-11-02T03:55:46.307ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 37b81eec-9fbb-4376-aec7-36daed3465bc, dataset_name: DatasetName { pool_name: ZpoolName { id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool), kind: External }, kind: Cockroach }, responder: ... }
7192024-11-02T03:55:46.324ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
7202024-11-02T03:55:46.324ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/6308f1ec-c08e-4282-a1c2-db37d6b8a044/config/omicron-zones.json file = common/src/ledger.rs:177
7212024-11-02T03:55:46.325ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/fda73319-9c6d-44d8-9377-b69537374c59/config/omicron-zones.json file = common/src/ledger.rs:177
7222024-11-02T03:55:46.325ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
7232024-11-02T03:55:46.356ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3277 zone = oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc
7242024-11-02T03:55:46.356ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3277 zone = oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2
7252024-11-02T03:55:46.356ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3277 zone = oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e
7262024-11-02T03:55:46.356ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3277 zone = oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360
7272024-11-02T03:55:46.356ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3277 zone = oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3
7282024-11-02T03:55:46.356ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc", TimeNotSynchronized), ("oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2", TimeNotSynchronized), ("oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e", TimeNotSynchronized), ("oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360", TimeNotSynchronized), ("oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:938 latency_us = 173711 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:59094 req_id = 473a81a3-45c8-4606-a570-365a6f609375 response_code = 500 uri = /omicron-zones
7292024-11-02T03:55:46.356ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "473a81a3-45c8-4606-a570-365a6f609375", "content-length": "124", "date": "Sat, 02 Nov 2024 03:55:45 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "473a81a3-45c8-4606-a570-365a6f609375" } file = sled-agent/src/rack_setup/service.rs:496 retry_after = 176.23799ms sled_address = [fd00:1122:3344:101::1]:12345
7302024-11-02T03:55:46.534ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 2c96eacb-1841-4277-9578-6b84caeef9c6 (service), filesystem_pool: Some(ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 1a7f019f-e208-4fae-b98e-fe6120f9b5e2 (service), filesystem_pool: Some(ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 0c3f42f2-ce05-428f-a91a-f78ae70ccccf (service), filesystem_pool: Some(ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 2d705ba5-6076-4a40-8774-81749916100b (service), filesystem_pool: Some(ZpoolName { id: 010339bd-18fb-4f1c-9455-7caf9e67e152 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 80c1df58-d201-4881-943a-0c6ca851a8d9, kind: Service { id: 2d705ba5-6076-4a40-8774-81749916100b }, name: Name("ntp-2d705ba5-6076-4a40-8774-81749916100b"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 255, 238])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: ea8c4184-9256-4357-bb1c-db39d54e6a2e (service), filesystem_pool: Some(ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External } } } }, OmicronZoneConfig { id: 41710ebd-a7df-4192-87ee-821f3687cee3 (service), filesystem_pool: Some(ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External } } } }, OmicronZoneConfig { id: 5b85d391-0238-40a9-b228-ddf2fb022360 (service), filesystem_pool: Some(ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External } } } }, OmicronZoneConfig { id: 4479233d-49d6-4267-9076-4d640d54f1e2 (service), filesystem_pool: Some(ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External } } } }, OmicronZoneConfig { id: 37b81eec-9fbb-4376-aec7-36daed3465bc (service), filesystem_pool: Some(ZpoolName { id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:459 sled_address = [fd00:1122:3344:101::1]:12345
7312024-11-02T03:55:46.534ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2c96eacb-1841-4277-9578-6b84caeef9c6, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
7322024-11-02T03:55:46.534ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 2c96eacb-1841-4277-9578-6b84caeef9c6, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: InternalDns }, responder: ... }
7332024-11-02T03:55:46.554ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 1a7f019f-e208-4fae-b98e-fe6120f9b5e2, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
7342024-11-02T03:55:46.554ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 1a7f019f-e208-4fae-b98e-fe6120f9b5e2, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: InternalDns }, responder: ... }
7352024-11-02T03:55:46.572ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 0c3f42f2-ce05-428f-a91a-f78ae70ccccf, dataset_name: DatasetName { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
7362024-11-02T03:55:46.572ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 0c3f42f2-ce05-428f-a91a-f78ae70ccccf, dataset_name: DatasetName { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }, kind: InternalDns }, responder: ... }
7372024-11-02T03:55:46.590ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ea8c4184-9256-4357-bb1c-db39d54e6a2e, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
7382024-11-02T03:55:46.590ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: ea8c4184-9256-4357-bb1c-db39d54e6a2e, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: Cockroach }, responder: ... }
7392024-11-02T03:55:46.608ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 41710ebd-a7df-4192-87ee-821f3687cee3, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
7402024-11-02T03:55:46.608ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 41710ebd-a7df-4192-87ee-821f3687cee3, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: Cockroach }, responder: ... }
7412024-11-02T03:55:46.627ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 5b85d391-0238-40a9-b228-ddf2fb022360, dataset_name: DatasetName { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
7422024-11-02T03:55:46.627ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 5b85d391-0238-40a9-b228-ddf2fb022360, dataset_name: DatasetName { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }, kind: Cockroach }, responder: ... }
7432024-11-02T03:55:46.644ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 4479233d-49d6-4267-9076-4d640d54f1e2, dataset_name: DatasetName { pool_name: ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
7442024-11-02T03:55:46.644ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 4479233d-49d6-4267-9076-4d640d54f1e2, dataset_name: DatasetName { pool_name: ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External }, kind: Cockroach }, responder: ... }
7452024-11-02T03:55:46.662ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 37b81eec-9fbb-4376-aec7-36daed3465bc, dataset_name: DatasetName { pool_name: ZpoolName { id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
7462024-11-02T03:55:46.662ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 37b81eec-9fbb-4376-aec7-36daed3465bc, dataset_name: DatasetName { pool_name: ZpoolName { id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool), kind: External }, kind: Cockroach }, responder: ... }
7472024-11-02T03:55:46.679ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
7482024-11-02T03:55:46.679ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/6308f1ec-c08e-4282-a1c2-db37d6b8a044/config/omicron-zones.json file = common/src/ledger.rs:177
7492024-11-02T03:55:46.680ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/fda73319-9c6d-44d8-9377-b69537374c59/config/omicron-zones.json file = common/src/ledger.rs:177
7502024-11-02T03:55:46.680ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
7512024-11-02T03:55:46.711ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3277 zone = oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc
7522024-11-02T03:55:46.711ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3277 zone = oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e
7532024-11-02T03:55:46.711ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3277 zone = oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3
7542024-11-02T03:55:46.711ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3277 zone = oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2
7552024-11-02T03:55:46.711ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3277 zone = oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360
7562024-11-02T03:55:46.712ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc", TimeNotSynchronized), ("oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e", TimeNotSynchronized), ("oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3", TimeNotSynchronized), ("oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2", TimeNotSynchronized), ("oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:938 latency_us = 176884 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:59094 req_id = 59a70fe2-47a7-41dc-9668-da38f2fc4d15 response_code = 500 uri = /omicron-zones
7572024-11-02T03:55:46.712ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "59a70fe2-47a7-41dc-9668-da38f2fc4d15", "content-length": "124", "date": "Sat, 02 Nov 2024 03:55:45 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "59a70fe2-47a7-41dc-9668-da38f2fc4d15" } file = sled-agent/src/rack_setup/service.rs:496 retry_after = 160.216285ms sled_address = [fd00:1122:3344:101::1]:12345
7582024-11-02T03:55:46.873ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 2c96eacb-1841-4277-9578-6b84caeef9c6 (service), filesystem_pool: Some(ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 1a7f019f-e208-4fae-b98e-fe6120f9b5e2 (service), filesystem_pool: Some(ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 0c3f42f2-ce05-428f-a91a-f78ae70ccccf (service), filesystem_pool: Some(ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 2d705ba5-6076-4a40-8774-81749916100b (service), filesystem_pool: Some(ZpoolName { id: 010339bd-18fb-4f1c-9455-7caf9e67e152 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 80c1df58-d201-4881-943a-0c6ca851a8d9, kind: Service { id: 2d705ba5-6076-4a40-8774-81749916100b }, name: Name("ntp-2d705ba5-6076-4a40-8774-81749916100b"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 255, 238])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: ea8c4184-9256-4357-bb1c-db39d54e6a2e (service), filesystem_pool: Some(ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External } } } }, OmicronZoneConfig { id: 41710ebd-a7df-4192-87ee-821f3687cee3 (service), filesystem_pool: Some(ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External } } } }, OmicronZoneConfig { id: 5b85d391-0238-40a9-b228-ddf2fb022360 (service), filesystem_pool: Some(ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External } } } }, OmicronZoneConfig { id: 4479233d-49d6-4267-9076-4d640d54f1e2 (service), filesystem_pool: Some(ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External } } } }, OmicronZoneConfig { id: 37b81eec-9fbb-4376-aec7-36daed3465bc (service), filesystem_pool: Some(ZpoolName { id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:459 sled_address = [fd00:1122:3344:101::1]:12345
7592024-11-02T03:55:46.874ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2c96eacb-1841-4277-9578-6b84caeef9c6, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
7602024-11-02T03:55:46.874ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 2c96eacb-1841-4277-9578-6b84caeef9c6, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: InternalDns }, responder: ... }
7612024-11-02T03:55:46.892ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 1a7f019f-e208-4fae-b98e-fe6120f9b5e2, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
7622024-11-02T03:55:46.892ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 1a7f019f-e208-4fae-b98e-fe6120f9b5e2, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: InternalDns }, responder: ... }
7632024-11-02T03:55:46.909ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 0c3f42f2-ce05-428f-a91a-f78ae70ccccf, dataset_name: DatasetName { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
7642024-11-02T03:55:46.909ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 0c3f42f2-ce05-428f-a91a-f78ae70ccccf, dataset_name: DatasetName { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }, kind: InternalDns }, responder: ... }
7652024-11-02T03:55:46.926ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ea8c4184-9256-4357-bb1c-db39d54e6a2e, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
7662024-11-02T03:55:46.926ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: ea8c4184-9256-4357-bb1c-db39d54e6a2e, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: Cockroach }, responder: ... }
7672024-11-02T03:55:46.943ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 41710ebd-a7df-4192-87ee-821f3687cee3, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
7682024-11-02T03:55:46.943ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 41710ebd-a7df-4192-87ee-821f3687cee3, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: Cockroach }, responder: ... }
7692024-11-02T03:55:46.962ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 5b85d391-0238-40a9-b228-ddf2fb022360, dataset_name: DatasetName { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
7702024-11-02T03:55:46.962ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 5b85d391-0238-40a9-b228-ddf2fb022360, dataset_name: DatasetName { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }, kind: Cockroach }, responder: ... }
7712024-11-02T03:55:46.980ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 4479233d-49d6-4267-9076-4d640d54f1e2, dataset_name: DatasetName { pool_name: ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
7722024-11-02T03:55:46.980ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 4479233d-49d6-4267-9076-4d640d54f1e2, dataset_name: DatasetName { pool_name: ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External }, kind: Cockroach }, responder: ... }
7732024-11-02T03:55:46.998ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 37b81eec-9fbb-4376-aec7-36daed3465bc, dataset_name: DatasetName { pool_name: ZpoolName { id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
7742024-11-02T03:55:46.998ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 37b81eec-9fbb-4376-aec7-36daed3465bc, dataset_name: DatasetName { pool_name: ZpoolName { id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool), kind: External }, kind: Cockroach }, responder: ... }
7752024-11-02T03:55:47.015ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
7762024-11-02T03:55:47.015ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/6308f1ec-c08e-4282-a1c2-db37d6b8a044/config/omicron-zones.json file = common/src/ledger.rs:177
7772024-11-02T03:55:47.016ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/fda73319-9c6d-44d8-9377-b69537374c59/config/omicron-zones.json file = common/src/ledger.rs:177
7782024-11-02T03:55:47.016ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
7792024-11-02T03:55:47.047ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3277 zone = oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3
7802024-11-02T03:55:47.047ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3277 zone = oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360
7812024-11-02T03:55:47.047ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3277 zone = oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2
7822024-11-02T03:55:47.047ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3277 zone = oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e
7832024-11-02T03:55:47.047ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3277 zone = oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc
7842024-11-02T03:55:47.047ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3", TimeNotSynchronized), ("oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360", TimeNotSynchronized), ("oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2", TimeNotSynchronized), ("oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e", TimeNotSynchronized), ("oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:938 latency_us = 173516 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:59094 req_id = beb3fcd2-af62-477f-a7e7-4bbb2dbfc103 response_code = 500 uri = /omicron-zones
7852024-11-02T03:55:47.048ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "beb3fcd2-af62-477f-a7e7-4bbb2dbfc103", "content-length": "124", "date": "Sat, 02 Nov 2024 03:55:47 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "beb3fcd2-af62-477f-a7e7-4bbb2dbfc103" } file = sled-agent/src/rack_setup/service.rs:496 retry_after = 241.901333ms sled_address = [fd00:1122:3344:101::1]:12345
7862024-11-02T03:55:47.292ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 2c96eacb-1841-4277-9578-6b84caeef9c6 (service), filesystem_pool: Some(ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 1a7f019f-e208-4fae-b98e-fe6120f9b5e2 (service), filesystem_pool: Some(ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 0c3f42f2-ce05-428f-a91a-f78ae70ccccf (service), filesystem_pool: Some(ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 2d705ba5-6076-4a40-8774-81749916100b (service), filesystem_pool: Some(ZpoolName { id: 010339bd-18fb-4f1c-9455-7caf9e67e152 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 80c1df58-d201-4881-943a-0c6ca851a8d9, kind: Service { id: 2d705ba5-6076-4a40-8774-81749916100b }, name: Name("ntp-2d705ba5-6076-4a40-8774-81749916100b"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 255, 238])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: ea8c4184-9256-4357-bb1c-db39d54e6a2e (service), filesystem_pool: Some(ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External } } } }, OmicronZoneConfig { id: 41710ebd-a7df-4192-87ee-821f3687cee3 (service), filesystem_pool: Some(ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External } } } }, OmicronZoneConfig { id: 5b85d391-0238-40a9-b228-ddf2fb022360 (service), filesystem_pool: Some(ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External } } } }, OmicronZoneConfig { id: 4479233d-49d6-4267-9076-4d640d54f1e2 (service), filesystem_pool: Some(ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External } } } }, OmicronZoneConfig { id: 37b81eec-9fbb-4376-aec7-36daed3465bc (service), filesystem_pool: Some(ZpoolName { id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:459 sled_address = [fd00:1122:3344:101::1]:12345
7872024-11-02T03:55:47.292ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2c96eacb-1841-4277-9578-6b84caeef9c6, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
7882024-11-02T03:55:47.292ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 2c96eacb-1841-4277-9578-6b84caeef9c6, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: InternalDns }, responder: ... }
7892024-11-02T03:55:47.312ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 1a7f019f-e208-4fae-b98e-fe6120f9b5e2, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
7902024-11-02T03:55:47.312ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 1a7f019f-e208-4fae-b98e-fe6120f9b5e2, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: InternalDns }, responder: ... }
7912024-11-02T03:55:47.330ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 0c3f42f2-ce05-428f-a91a-f78ae70ccccf, dataset_name: DatasetName { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
7922024-11-02T03:55:47.330ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 0c3f42f2-ce05-428f-a91a-f78ae70ccccf, dataset_name: DatasetName { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }, kind: InternalDns }, responder: ... }
7932024-11-02T03:55:47.348ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ea8c4184-9256-4357-bb1c-db39d54e6a2e, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
7942024-11-02T03:55:47.348ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: ea8c4184-9256-4357-bb1c-db39d54e6a2e, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: Cockroach }, responder: ... }
7952024-11-02T03:55:47.366ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 41710ebd-a7df-4192-87ee-821f3687cee3, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
7962024-11-02T03:55:47.366ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 41710ebd-a7df-4192-87ee-821f3687cee3, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: Cockroach }, responder: ... }
7972024-11-02T03:55:47.384ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 5b85d391-0238-40a9-b228-ddf2fb022360, dataset_name: DatasetName { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
7982024-11-02T03:55:47.384ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 5b85d391-0238-40a9-b228-ddf2fb022360, dataset_name: DatasetName { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }, kind: Cockroach }, responder: ... }
7992024-11-02T03:55:47.402ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 4479233d-49d6-4267-9076-4d640d54f1e2, dataset_name: DatasetName { pool_name: ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
8002024-11-02T03:55:47.402ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 4479233d-49d6-4267-9076-4d640d54f1e2, dataset_name: DatasetName { pool_name: ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8012024-11-02T03:55:47.419ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 37b81eec-9fbb-4376-aec7-36daed3465bc, dataset_name: DatasetName { pool_name: ZpoolName { id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
8022024-11-02T03:55:47.419ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 37b81eec-9fbb-4376-aec7-36daed3465bc, dataset_name: DatasetName { pool_name: ZpoolName { id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8032024-11-02T03:55:47.438ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8042024-11-02T03:55:47.438ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/6308f1ec-c08e-4282-a1c2-db37d6b8a044/config/omicron-zones.json file = common/src/ledger.rs:177
8052024-11-02T03:55:47.438ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/fda73319-9c6d-44d8-9377-b69537374c59/config/omicron-zones.json file = common/src/ledger.rs:177
8062024-11-02T03:55:47.438ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8072024-11-02T03:55:47.470ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3277 zone = oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc
8082024-11-02T03:55:47.470ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3277 zone = oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2
8092024-11-02T03:55:47.470ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3277 zone = oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360
8102024-11-02T03:55:47.470ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3277 zone = oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e
8112024-11-02T03:55:47.470ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3277 zone = oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3
8122024-11-02T03:55:47.470ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc", TimeNotSynchronized), ("oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2", TimeNotSynchronized), ("oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360", TimeNotSynchronized), ("oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e", TimeNotSynchronized), ("oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:938 latency_us = 177353 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:59094 req_id = 39b31559-7f78-4a50-975c-c352f40b0d00 response_code = 500 uri = /omicron-zones
8132024-11-02T03:55:47.470ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "39b31559-7f78-4a50-975c-c352f40b0d00", "content-length": "124", "date": "Sat, 02 Nov 2024 03:55:46 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "39b31559-7f78-4a50-975c-c352f40b0d00" } file = sled-agent/src/rack_setup/service.rs:496 retry_after = 150.682454ms sled_address = [fd00:1122:3344:101::1]:12345
8142024-11-02T03:55:47.623ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 2c96eacb-1841-4277-9578-6b84caeef9c6 (service), filesystem_pool: Some(ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 1a7f019f-e208-4fae-b98e-fe6120f9b5e2 (service), filesystem_pool: Some(ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 0c3f42f2-ce05-428f-a91a-f78ae70ccccf (service), filesystem_pool: Some(ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 2d705ba5-6076-4a40-8774-81749916100b (service), filesystem_pool: Some(ZpoolName { id: 010339bd-18fb-4f1c-9455-7caf9e67e152 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 80c1df58-d201-4881-943a-0c6ca851a8d9, kind: Service { id: 2d705ba5-6076-4a40-8774-81749916100b }, name: Name("ntp-2d705ba5-6076-4a40-8774-81749916100b"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 255, 238])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: ea8c4184-9256-4357-bb1c-db39d54e6a2e (service), filesystem_pool: Some(ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External } } } }, OmicronZoneConfig { id: 41710ebd-a7df-4192-87ee-821f3687cee3 (service), filesystem_pool: Some(ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External } } } }, OmicronZoneConfig { id: 5b85d391-0238-40a9-b228-ddf2fb022360 (service), filesystem_pool: Some(ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External } } } }, OmicronZoneConfig { id: 4479233d-49d6-4267-9076-4d640d54f1e2 (service), filesystem_pool: Some(ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External } } } }, OmicronZoneConfig { id: 37b81eec-9fbb-4376-aec7-36daed3465bc (service), filesystem_pool: Some(ZpoolName { id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:459 sled_address = [fd00:1122:3344:101::1]:12345
8152024-11-02T03:55:47.623ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2c96eacb-1841-4277-9578-6b84caeef9c6, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
8162024-11-02T03:55:47.623ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 2c96eacb-1841-4277-9578-6b84caeef9c6, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: InternalDns }, responder: ... }
8172024-11-02T03:55:47.645ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 1a7f019f-e208-4fae-b98e-fe6120f9b5e2, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
8182024-11-02T03:55:47.645ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 1a7f019f-e208-4fae-b98e-fe6120f9b5e2, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: InternalDns }, responder: ... }
8192024-11-02T03:55:47.663ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 0c3f42f2-ce05-428f-a91a-f78ae70ccccf, dataset_name: DatasetName { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
8202024-11-02T03:55:47.663ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 0c3f42f2-ce05-428f-a91a-f78ae70ccccf, dataset_name: DatasetName { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }, kind: InternalDns }, responder: ... }
8212024-11-02T03:55:47.682ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ea8c4184-9256-4357-bb1c-db39d54e6a2e, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
8222024-11-02T03:55:47.682ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: ea8c4184-9256-4357-bb1c-db39d54e6a2e, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: Cockroach }, responder: ... }
8232024-11-02T03:55:47.700ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 41710ebd-a7df-4192-87ee-821f3687cee3, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
8242024-11-02T03:55:47.700ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 41710ebd-a7df-4192-87ee-821f3687cee3, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8252024-11-02T03:55:47.718ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 5b85d391-0238-40a9-b228-ddf2fb022360, dataset_name: DatasetName { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
8262024-11-02T03:55:47.718ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 5b85d391-0238-40a9-b228-ddf2fb022360, dataset_name: DatasetName { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8272024-11-02T03:55:47.736ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 4479233d-49d6-4267-9076-4d640d54f1e2, dataset_name: DatasetName { pool_name: ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
8282024-11-02T03:55:47.736ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 4479233d-49d6-4267-9076-4d640d54f1e2, dataset_name: DatasetName { pool_name: ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8292024-11-02T03:55:47.755ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 37b81eec-9fbb-4376-aec7-36daed3465bc, dataset_name: DatasetName { pool_name: ZpoolName { id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
8302024-11-02T03:55:47.755ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 37b81eec-9fbb-4376-aec7-36daed3465bc, dataset_name: DatasetName { pool_name: ZpoolName { id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8312024-11-02T03:55:47.773ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8322024-11-02T03:55:47.773ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/6308f1ec-c08e-4282-a1c2-db37d6b8a044/config/omicron-zones.json file = common/src/ledger.rs:177
8332024-11-02T03:55:47.773ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/fda73319-9c6d-44d8-9377-b69537374c59/config/omicron-zones.json file = common/src/ledger.rs:177
8342024-11-02T03:55:47.773ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8352024-11-02T03:55:47.806ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3277 zone = oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360
8362024-11-02T03:55:47.806ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3277 zone = oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2
8372024-11-02T03:55:47.806ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3277 zone = oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc
8382024-11-02T03:55:47.806ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3277 zone = oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3
8392024-11-02T03:55:47.806ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3277 zone = oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e
8402024-11-02T03:55:47.806ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360", TimeNotSynchronized), ("oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2", TimeNotSynchronized), ("oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc", TimeNotSynchronized), ("oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3", TimeNotSynchronized), ("oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:938 latency_us = 182588 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:59094 req_id = 3cc32c3c-579d-4633-8681-85ce0c4994c4 response_code = 500 uri = /omicron-zones
8412024-11-02T03:55:47.806ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "3cc32c3c-579d-4633-8681-85ce0c4994c4", "content-length": "124", "date": "Sat, 02 Nov 2024 03:55:47 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "3cc32c3c-579d-4633-8681-85ce0c4994c4" } file = sled-agent/src/rack_setup/service.rs:496 retry_after = 176.046026ms sled_address = [fd00:1122:3344:101::1]:12345
8422024-11-02T03:55:47.983ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 2c96eacb-1841-4277-9578-6b84caeef9c6 (service), filesystem_pool: Some(ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 1a7f019f-e208-4fae-b98e-fe6120f9b5e2 (service), filesystem_pool: Some(ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 0c3f42f2-ce05-428f-a91a-f78ae70ccccf (service), filesystem_pool: Some(ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 2d705ba5-6076-4a40-8774-81749916100b (service), filesystem_pool: Some(ZpoolName { id: 010339bd-18fb-4f1c-9455-7caf9e67e152 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 80c1df58-d201-4881-943a-0c6ca851a8d9, kind: Service { id: 2d705ba5-6076-4a40-8774-81749916100b }, name: Name("ntp-2d705ba5-6076-4a40-8774-81749916100b"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 255, 238])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: ea8c4184-9256-4357-bb1c-db39d54e6a2e (service), filesystem_pool: Some(ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External } } } }, OmicronZoneConfig { id: 41710ebd-a7df-4192-87ee-821f3687cee3 (service), filesystem_pool: Some(ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External } } } }, OmicronZoneConfig { id: 5b85d391-0238-40a9-b228-ddf2fb022360 (service), filesystem_pool: Some(ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External } } } }, OmicronZoneConfig { id: 4479233d-49d6-4267-9076-4d640d54f1e2 (service), filesystem_pool: Some(ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External } } } }, OmicronZoneConfig { id: 37b81eec-9fbb-4376-aec7-36daed3465bc (service), filesystem_pool: Some(ZpoolName { id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:459 sled_address = [fd00:1122:3344:101::1]:12345
8432024-11-02T03:55:47.983ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2c96eacb-1841-4277-9578-6b84caeef9c6, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
8442024-11-02T03:55:47.983ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 2c96eacb-1841-4277-9578-6b84caeef9c6, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: InternalDns }, responder: ... }
8452024-11-02T03:55:48.003ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 1a7f019f-e208-4fae-b98e-fe6120f9b5e2, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
8462024-11-02T03:55:48.003ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 1a7f019f-e208-4fae-b98e-fe6120f9b5e2, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: InternalDns }, responder: ... }
8472024-11-02T03:55:48.020ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 0c3f42f2-ce05-428f-a91a-f78ae70ccccf, dataset_name: DatasetName { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
8482024-11-02T03:55:48.020ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 0c3f42f2-ce05-428f-a91a-f78ae70ccccf, dataset_name: DatasetName { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }, kind: InternalDns }, responder: ... }
8492024-11-02T03:55:48.037ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ea8c4184-9256-4357-bb1c-db39d54e6a2e, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
8502024-11-02T03:55:48.037ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: ea8c4184-9256-4357-bb1c-db39d54e6a2e, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: Cockroach }, responder: ... }
8512024-11-02T03:55:48.056ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 41710ebd-a7df-4192-87ee-821f3687cee3, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
8522024-11-02T03:55:48.056ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 41710ebd-a7df-4192-87ee-821f3687cee3, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8532024-11-02T03:55:48.073ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 5b85d391-0238-40a9-b228-ddf2fb022360, dataset_name: DatasetName { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
8542024-11-02T03:55:48.073ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 5b85d391-0238-40a9-b228-ddf2fb022360, dataset_name: DatasetName { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8552024-11-02T03:55:48.090ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 4479233d-49d6-4267-9076-4d640d54f1e2, dataset_name: DatasetName { pool_name: ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
8562024-11-02T03:55:48.090ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 4479233d-49d6-4267-9076-4d640d54f1e2, dataset_name: DatasetName { pool_name: ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8572024-11-02T03:55:48.107ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 37b81eec-9fbb-4376-aec7-36daed3465bc, dataset_name: DatasetName { pool_name: ZpoolName { id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
8582024-11-02T03:55:48.107ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 37b81eec-9fbb-4376-aec7-36daed3465bc, dataset_name: DatasetName { pool_name: ZpoolName { id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8592024-11-02T03:55:48.124ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8602024-11-02T03:55:48.124ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/6308f1ec-c08e-4282-a1c2-db37d6b8a044/config/omicron-zones.json file = common/src/ledger.rs:177
8612024-11-02T03:55:48.124ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/fda73319-9c6d-44d8-9377-b69537374c59/config/omicron-zones.json file = common/src/ledger.rs:177
8622024-11-02T03:55:48.124ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8632024-11-02T03:55:48.158ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3277 zone = oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360
8642024-11-02T03:55:48.158ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3277 zone = oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2
8652024-11-02T03:55:48.158ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3277 zone = oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e
8662024-11-02T03:55:48.158ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3277 zone = oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc
8672024-11-02T03:55:48.158ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3277 zone = oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3
8682024-11-02T03:55:48.158ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360", TimeNotSynchronized), ("oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2", TimeNotSynchronized), ("oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e", TimeNotSynchronized), ("oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc", TimeNotSynchronized), ("oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:938 latency_us = 174421 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:59094 req_id = 5d7a56c0-124a-4935-998a-08ad83fda51f response_code = 500 uri = /omicron-zones
8692024-11-02T03:55:48.158ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "5d7a56c0-124a-4935-998a-08ad83fda51f", "content-length": "124", "date": "Sat, 02 Nov 2024 03:55:47 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "5d7a56c0-124a-4935-998a-08ad83fda51f" } file = sled-agent/src/rack_setup/service.rs:496 retry_after = 222.440995ms sled_address = [fd00:1122:3344:101::1]:12345
8702024-11-02T03:55:48.382ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 2c96eacb-1841-4277-9578-6b84caeef9c6 (service), filesystem_pool: Some(ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 1a7f019f-e208-4fae-b98e-fe6120f9b5e2 (service), filesystem_pool: Some(ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 0c3f42f2-ce05-428f-a91a-f78ae70ccccf (service), filesystem_pool: Some(ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 2d705ba5-6076-4a40-8774-81749916100b (service), filesystem_pool: Some(ZpoolName { id: 010339bd-18fb-4f1c-9455-7caf9e67e152 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 80c1df58-d201-4881-943a-0c6ca851a8d9, kind: Service { id: 2d705ba5-6076-4a40-8774-81749916100b }, name: Name("ntp-2d705ba5-6076-4a40-8774-81749916100b"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 255, 238])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: ea8c4184-9256-4357-bb1c-db39d54e6a2e (service), filesystem_pool: Some(ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External } } } }, OmicronZoneConfig { id: 41710ebd-a7df-4192-87ee-821f3687cee3 (service), filesystem_pool: Some(ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External } } } }, OmicronZoneConfig { id: 5b85d391-0238-40a9-b228-ddf2fb022360 (service), filesystem_pool: Some(ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External } } } }, OmicronZoneConfig { id: 4479233d-49d6-4267-9076-4d640d54f1e2 (service), filesystem_pool: Some(ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External } } } }, OmicronZoneConfig { id: 37b81eec-9fbb-4376-aec7-36daed3465bc (service), filesystem_pool: Some(ZpoolName { id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:459 sled_address = [fd00:1122:3344:101::1]:12345
8712024-11-02T03:55:48.382ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2c96eacb-1841-4277-9578-6b84caeef9c6, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
8722024-11-02T03:55:48.382ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 2c96eacb-1841-4277-9578-6b84caeef9c6, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: InternalDns }, responder: ... }
8732024-11-02T03:55:48.401ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 1a7f019f-e208-4fae-b98e-fe6120f9b5e2, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
8742024-11-02T03:55:48.401ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 1a7f019f-e208-4fae-b98e-fe6120f9b5e2, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: InternalDns }, responder: ... }
8752024-11-02T03:55:48.418ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 0c3f42f2-ce05-428f-a91a-f78ae70ccccf, dataset_name: DatasetName { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
8762024-11-02T03:55:48.418ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 0c3f42f2-ce05-428f-a91a-f78ae70ccccf, dataset_name: DatasetName { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }, kind: InternalDns }, responder: ... }
8772024-11-02T03:55:48.436ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ea8c4184-9256-4357-bb1c-db39d54e6a2e, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
8782024-11-02T03:55:48.436ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: ea8c4184-9256-4357-bb1c-db39d54e6a2e, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: Cockroach }, responder: ... }
8792024-11-02T03:55:48.455ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 41710ebd-a7df-4192-87ee-821f3687cee3, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
8802024-11-02T03:55:48.455ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 41710ebd-a7df-4192-87ee-821f3687cee3, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8812024-11-02T03:55:48.473ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 5b85d391-0238-40a9-b228-ddf2fb022360, dataset_name: DatasetName { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
8822024-11-02T03:55:48.473ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 5b85d391-0238-40a9-b228-ddf2fb022360, dataset_name: DatasetName { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8832024-11-02T03:55:48.492ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 4479233d-49d6-4267-9076-4d640d54f1e2, dataset_name: DatasetName { pool_name: ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
8842024-11-02T03:55:48.492ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 4479233d-49d6-4267-9076-4d640d54f1e2, dataset_name: DatasetName { pool_name: ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8852024-11-02T03:55:48.509ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 280.800973ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:424 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
8862024-11-02T03:55:48.510ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 37b81eec-9fbb-4376-aec7-36daed3465bc, dataset_name: DatasetName { pool_name: ZpoolName { id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
8872024-11-02T03:55:48.510ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 37b81eec-9fbb-4376-aec7-36daed3465bc, dataset_name: DatasetName { pool_name: ZpoolName { id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8882024-11-02T03:55:48.528ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8892024-11-02T03:55:48.528ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/6308f1ec-c08e-4282-a1c2-db37d6b8a044/config/omicron-zones.json file = common/src/ledger.rs:177
8902024-11-02T03:55:48.528ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/fda73319-9c6d-44d8-9377-b69537374c59/config/omicron-zones.json file = common/src/ledger.rs:177
8912024-11-02T03:55:48.528ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8922024-11-02T03:55:48.612ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8932024-11-02T03:55:48.612ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8942024-11-02T03:55:48.612ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8952024-11-02T03:55:48.612ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8962024-11-02T03:55:48.612ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
8972024-11-02T03:55:48.706ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc file = illumos-utils/src/zone.rs:325
8982024-11-02T03:55:48.710ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2 file = illumos-utils/src/zone.rs:325
8992024-11-02T03:55:48.714ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360 file = illumos-utils/src/zone.rs:325
9002024-11-02T03:55:48.719ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e file = illumos-utils/src/zone.rs:325
9012024-11-02T03:55:48.723ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3 file = illumos-utils/src/zone.rs:325
9022024-11-02T03:55:48.728ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc file = illumos-utils/src/zone.rs:363
9032024-11-02T03:55:48.732ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2 file = illumos-utils/src/zone.rs:363
9042024-11-02T03:55:48.737ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360 file = illumos-utils/src/zone.rs:363
9052024-11-02T03:55:48.741ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e file = illumos-utils/src/zone.rs:363
9062024-11-02T03:55:48.745ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3 file = illumos-utils/src/zone.rs:363
9072024-11-02T03:55:51.581ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService5'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::7'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::7]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='37b81eec-9fbb-4376-aec7-36daed3465bc'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::7]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::7]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9082024-11-02T03:55:51.582ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc
9092024-11-02T03:55:51.587ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService6'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::6]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='4479233d-49d6-4267-9076-4d640d54f1e2'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::6]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::6]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9102024-11-02T03:55:51.587ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2
9112024-11-02T03:55:51.665ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService9'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::4'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::4]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='41710ebd-a7df-4192-87ee-821f3687cee3'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::4]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::4]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9122024-11-02T03:55:51.665ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3
9132024-11-02T03:55:51.675ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService7'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::5]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='5b85d391-0238-40a9-b228-ddf2fb022360'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::5]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::5]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9142024-11-02T03:55:51.675ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360
9152024-11-02T03:55:51.721ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc") failed: Property not found. retry in 40.063313ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc
9162024-11-02T03:55:51.721ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService8'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::3'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::3]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='ea8c4184-9256-4357-bb1c-db39d54e6a2e'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::3]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::3]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9172024-11-02T03:55:51.721ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e
9182024-11-02T03:55:51.733ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2") failed: Property not found. retry in 68.665653ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2
9192024-11-02T03:55:51.769ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc") failed: Property not found. retry in 142.616313ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc
9202024-11-02T03:55:51.908ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2") failed: Property not found. retry in 145.561227ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2
9212024-11-02T03:55:52.370ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc") failed: Property not found. retry in 234.103965ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc
9222024-11-02T03:55:52.420ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2") failed: Property not found. retry in 229.45316ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2
9232024-11-02T03:55:52.455ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360") failed: Property not found. retry in 28.320987ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360
9242024-11-02T03:55:52.485ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3") failed: Property not found. retry in 36.407344ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3
9252024-11-02T03:55:52.518ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360") failed: Property not found. retry in 77.976179ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360
9262024-11-02T03:55:52.553ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e") failed: Property not found. retry in 67.615019ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e
9272024-11-02T03:55:52.574ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3") failed: Property not found. retry in 144.075493ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3
9282024-11-02T03:55:52.632ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360") failed: Property not found. retry in 160.431503ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360
9292024-11-02T03:55:52.668ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc") failed: Property not found. retry in 407.338872ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc
9302024-11-02T03:55:52.692ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2") failed: Property not found. retry in 529.580564ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2
9312024-11-02T03:55:52.798ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e") failed: Property not found. retry in 76.17792ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e
9322024-11-02T03:55:52.813ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3") failed: Property not found. retry in 168.95514ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3
9332024-11-02T03:55:52.920ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360") failed: Property not found. retry in 581.973712ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360
9342024-11-02T03:55:53.099ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e") failed: Property not found. retry in 187.490254ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e
9352024-11-02T03:55:53.127ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3") failed: Property not found. retry in 457.482187ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3
9362024-11-02T03:55:53.237ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc") failed: Property not found. retry in 1.154653048s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc
9372024-11-02T03:55:53.274ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2") failed: Property not found. retry in 708.680026ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2
9382024-11-02T03:55:53.381ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e") failed: Property not found. retry in 411.221873ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e
9392024-11-02T03:55:53.519ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360") failed: Property not found. retry in 425.730285ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360
9402024-11-02T03:55:53.605ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3") failed: Property not found. retry in 700.008172ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3
9412024-11-02T03:55:53.816ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e") failed: Property not found. retry in 815.904029ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e
9422024-11-02T03:55:53.964ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360") failed: Property not found. retry in 959.007912ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360
9432024-11-02T03:55:54.008ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2") failed: Property not found. retry in 897.216159ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2
9442024-11-02T03:55:54.319ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3") failed: Property not found. retry in 1.170178358s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3
9452024-11-02T03:55:54.405ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc") failed: Property not found. retry in 567.481594ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc
9462024-11-02T03:55:54.644ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e") failed: Property not found. retry in 658.398283ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e
9472024-11-02T03:55:54.916ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2") failed: Property not found. retry in 747.388437ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2
9482024-11-02T03:55:54.936ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360") failed: Property not found. retry in 1.140530783s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360
9492024-11-02T03:55:54.981ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc") failed: Property not found. retry in 550.403851ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc
9502024-11-02T03:55:55.313ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e") failed: Property not found. retry in 1.02300334s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e
9512024-11-02T03:55:55.500ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3") failed: Property not found. retry in 1.295788065s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3
9522024-11-02T03:55:55.543ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc") failed: Property not found. retry in 701.346049ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc
9532024-11-02T03:55:55.674ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2") failed: Property not found. retry in 1.095095758s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2
9542024-11-02T03:55:56.089ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360") failed: Property not found. retry in 990.421495ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360
9552024-11-02T03:55:56.256ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc") failed: Property not found. retry in 1.245767101s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc
9562024-11-02T03:55:56.347ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e") failed: Property not found. retry in 1.089870654s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e
9572024-11-02T03:55:56.782ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2") failed: Property not found. retry in 527.97211ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2
9582024-11-02T03:55:56.809ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3") failed: Property not found. retry in 870.436357ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3
9592024-11-02T03:55:57.091ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360") failed: Property not found. retry in 781.55079ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360
9602024-11-02T03:55:57.321ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2") failed: Property not found. retry in 954.782746ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2
9612024-11-02T03:55:57.449ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e") failed: Property not found. retry in 1.185827083s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e
9622024-11-02T03:55:57.514ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc") failed: Property not found. retry in 783.029334ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc
9632024-11-02T03:55:57.690ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3") failed: Property not found. retry in 791.217822ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3
9642024-11-02T03:55:57.884ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360") failed: Property not found. retry in 1.3811521s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360
9652024-11-02T03:55:58.288ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2") failed: Property not found. retry in 839.302806ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2
9662024-11-02T03:55:58.309ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc") failed: Property not found. retry in 1.168147731s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc
9672024-11-02T03:55:58.492ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3") failed: Property not found. retry in 1.086949026s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3
9682024-11-02T03:55:58.647ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e") failed: Property not found. retry in 1.072600925s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e
9692024-11-02T03:55:59.140ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2") failed: Property not found. retry in 827.685966ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2
9702024-11-02T03:55:59.279ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360") failed: Property not found. retry in 1.182001784s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360
9712024-11-02T03:55:59.454ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:50874
9722024-11-02T03:55:59.455ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 90 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:50874 req_id = 45c2f16a-08b5-4d34-b94b-751e034a8e94 response_code = 200 uri = /baseboard
9732024-11-02T03:55:59.489ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc") failed: Property not found. retry in 677.184646ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc
9742024-11-02T03:55:59.590ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3") failed: Property not found. retry in 1.175001986s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3
9752024-11-02T03:55:59.731ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e") failed: Property not found. retry in 1.168634255s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e
9762024-11-02T03:55:59.979ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2") failed: Property not found. retry in 1.246915009s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2
9772024-11-02T03:56:00.177ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc") failed: Property not found. retry in 1.464551726s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc
9782024-11-02T03:56:00.472ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360") failed: Property not found. retry in 1.255180745s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360
9792024-11-02T03:56:00.778ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3") failed: Property not found. retry in 829.802383ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3
9802024-11-02T03:56:00.913ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e") failed: Property not found. retry in 957.003718ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e
9812024-11-02T03:56:01.237ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2") failed: Property not found. retry in 501.897952ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2
9822024-11-02T03:56:01.618ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3") failed: Property not found. retry in 629.006373ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3
9832024-11-02T03:56:01.653ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc") failed: Property not found. retry in 1.436174542s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc
9842024-11-02T03:56:01.737ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360") failed: Property not found. retry in 1.04085074s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360
9852024-11-02T03:56:01.747ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2") failed: Property not found. retry in 1.163324365s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2
9862024-11-02T03:56:01.879ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e") failed: Property not found. retry in 857.367627ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e
9872024-11-02T03:56:02.255ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3") failed: Property not found. retry in 1.427425496s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3
9882024-11-02T03:56:02.746ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e") failed: Property not found. retry in 789.091948ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e
9892024-11-02T03:56:02.787ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360") failed: Property not found. retry in 1.084459358s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360
9902024-11-02T03:56:02.919ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2") failed: Property not found. retry in 1.092560446s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2
9912024-11-02T03:56:03.098ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc") failed: Property not found. retry in 800.137301ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc
9922024-11-02T03:56:03.546ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e") failed: Property not found. retry in 1.494684489s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e
9932024-11-02T03:56:03.692ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3") failed: Property not found. retry in 668.612171ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3
9942024-11-02T03:56:03.870ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 499.346788ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:424 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
9952024-11-02T03:56:03.883ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360") failed: Property not found. retry in 1.290770395s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360
9962024-11-02T03:56:03.907ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc") failed: Property not found. retry in 1.103798669s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc
9972024-11-02T03:56:04.020ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2") failed: Property not found. retry in 1.026678811s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2
9982024-11-02T03:56:04.370ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3") failed: Property not found. retry in 814.977317ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3
9992024-11-02T03:56:05.022ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc") failed: Property not found. retry in 592.957442ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc
10002024-11-02T03:56:05.049ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e") failed: Property not found. retry in 796.325353ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e
10012024-11-02T03:56:05.057ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2") failed: Property not found. retry in 628.657885ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2
10022024-11-02T03:56:05.182ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360") failed: Property not found. retry in 1.355655927s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360
10032024-11-02T03:56:05.194ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3") failed: Property not found. retry in 1.363742756s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3
10042024-11-02T03:56:05.624ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc") failed: Property not found. retry in 619.864392ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc
10052024-11-02T03:56:05.695ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2") failed: Property not found. retry in 1.255478039s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2
10062024-11-02T03:56:05.855ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e") failed: Property not found. retry in 1.425966079s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e
10072024-11-02T03:56:06.253ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc") failed: Property not found. retry in 828.414519ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc
10082024-11-02T03:56:06.546ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360") failed: Property not found. retry in 1.151197052s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360
10092024-11-02T03:56:06.566ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3") failed: Property not found. retry in 1.452891041s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3
10102024-11-02T03:56:06.960ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2") failed: Property not found. retry in 1.101847841s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2
10112024-11-02T03:56:07.091ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc") failed: Property not found. retry in 974.317768ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc
10122024-11-02T03:56:08.100ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3273 zone = oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc
10132024-11-02T03:56:08.100ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3273 zone = oxz_cockroachdb_4479233d-49d6-4267-9076-4d640d54f1e2
10142024-11-02T03:56:08.100ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3273 zone = oxz_cockroachdb_5b85d391-0238-40a9-b228-ddf2fb022360
10152024-11-02T03:56:08.100ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3273 zone = oxz_cockroachdb_ea8c4184-9256-4357-bb1c-db39d54e6a2e
10162024-11-02T03:56:08.100ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3273 zone = oxz_cockroachdb_41710ebd-a7df-4192-87ee-821f3687cee3
10172024-11-02T03:56:08.100ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/6308f1ec-c08e-4282-a1c2-db37d6b8a044/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
10182024-11-02T03:56:08.100ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/fda73319-9c6d-44d8-9377-b69537374c59/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
10192024-11-02T03:56:08.100ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 19681776 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:59094 req_id = d94004ba-e74c-42a1-b7d5-8ed8c4a544c3 response_code = 204 uri = /omicron-zones
10202024-11-02T03:56:08.135ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:46329
10212024-11-02T03:56:08.163ZINFOSledAgent (ServiceManager): Initializing CRDB Cluster - sending request to [fd00:1122:3344:101::7]:32221 file = sled-agent/src/services.rs:3705
10222024-11-02T03:56:23.180ZINFOSledAgent (ServiceManager): Formatting CRDB file = sled-agent/src/services.rs:3723
10232024-11-02T03:56:24.871ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize CockroachDb: Error running command in zone 'oxz_cockroachdb_37b81eec-9fbb-4376-aec7-36daed3465bc': Command [/opt/oxide/cockroachdb/bin/cockroach sql --insecure --host [fd00:1122:3344:101::7]:32221 --file /opt/oxide/cockroachdb/sql/dbinit.sql] executed and failed with status: exit status: 1 stdout: BEGIN\nCREATE DATABASE\nCREATE ROLE\nALTER DEFAULT PRIVILEGES\nCONFIGURE ZONE 1\nCREATE TYPE\nCREATE TABLE\nCREATE TABLE\nCREATE TYPE\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE TYPE\nCREATE TYPE\nCREATE TYPE\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE TABLE\nCREATE TABLE\nCREATE TABLE\nCREATE INDEX\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE TYPE\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE TABLE\nCREATE VIEW\nCREATE TABLE\nCREATE INDEX\nCREATE TYPE\nCREATE TYPE\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE VIEW\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE TABLE\nCREATE VIEW\nCREATE VIEW\nCREATE INDEX\nCREATE INDEX\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE VIEW\nCREATE VIEW\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE TYPE\nCREATE TYPE\nCREATE TYPE\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE TYPE\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE INDEX\nCREATE VIEW\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE TABLE\nCREATE TABLE\nCREATE TABLE\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nINSERT 1\nCREATE TABLE\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE TABLE\nCREATE TYPE\nCREATE TABLE\nCREATE TYPE\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE TABLE\nCREATE INDEX\nCREATE TABLE\nCREATE INDEX\nCREATE TYPE\nCREATE TABLE\nCREATE TYPE\nCREATE TYPE\nCREATE TABLE\nCREATE TABLE\n stderr: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionAbortedError(ABORT_REASON_ABORT_SPAN): "sql txn" meta={id=705fd73b key=/NamespaceTable/30/1/106/0/"public"/4/1 pri=0.00291420 epo=0 ts=1730519784.746008561,1 min=1730519783.870336726,0 seq=1652} lock=true stat=ABORTED rts=1730519783.870336726,0 wto=false gul=1730519784.370336726,0\nSQLSTATE: 40001\nHINT: See: https://www.cockroachlabs.com/docs/v22.1/transaction-retry-error-reference.html#abort_reason_abort_span\nERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionAbortedError(ABORT_REASON_ABORT_SPAN): "sql txn" meta={id=705fd73b key=/NamespaceTable/30/1/106/0/"public"/4/1 pri=0.00291420 epo=0 ts=1730519784.746008561,1 min=1730519783.870336726,0 seq=1652} lock=true stat=ABORTED rts=1730519783.870336726,0 wto=false gul=1730519784.370336726,0\nSQLSTATE: 40001\nHINT: See: https://www.cockroachlabs.com/docs/v22.1/transaction-retry-error-reference.html#abort_reason_abort_span\nFailed running "sql"\n file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:938 latency_us = 16735606 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:46329 req_id = 5af1b8d3-c3f1-4cdb-8ab7-e2326ebf1baa response_code = 500 uri = /cockroachdb
10242024-11-02T03:56:24.872ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "5af1b8d3-c3f1-4cdb-8ab7-e2326ebf1baa", "content-length": "124", "date": "Sat, 02 Nov 2024 03:56:24 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "5af1b8d3-c3f1-4cdb-8ab7-e2326ebf1baa" } file = sled-agent/src/rack_setup/service.rs:1037 retry_after = 54.984478ms
10252024-11-02T03:56:24.878ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 1.337534928s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:424 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
10262024-11-02T03:56:24.963ZINFOSledAgent (ServiceManager): Initializing CRDB Cluster - sending request to [fd00:1122:3344:101::7]:32221 file = sled-agent/src/services.rs:3705
10272024-11-02T03:56:25.077ZINFOSledAgent (ServiceManager): Formatting CRDB file = sled-agent/src/services.rs:3723
10282024-11-02T03:56:28.336ZINFOSledAgent (ServiceManager): Formatting CRDB - Completed file = sled-agent/src/services.rs:3746
10292024-11-02T03:56:28.336ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 3408574 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:46329 req_id = 0a6bc315-ebcc-4251-9578-3d093281593b response_code = 204 uri = /cockroachdb
10302024-11-02T03:56:28.409ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(5), zones: [OmicronZoneConfig { id: 2c96eacb-1841-4277-9578-6b84caeef9c6 (service), filesystem_pool: Some(ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 1a7f019f-e208-4fae-b98e-fe6120f9b5e2 (service), filesystem_pool: Some(ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 0c3f42f2-ce05-428f-a91a-f78ae70ccccf (service), filesystem_pool: Some(ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 2d705ba5-6076-4a40-8774-81749916100b (service), filesystem_pool: Some(ZpoolName { id: 010339bd-18fb-4f1c-9455-7caf9e67e152 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 80c1df58-d201-4881-943a-0c6ca851a8d9, kind: Service { id: 2d705ba5-6076-4a40-8774-81749916100b }, name: Name("ntp-2d705ba5-6076-4a40-8774-81749916100b"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 255, 238])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: ea8c4184-9256-4357-bb1c-db39d54e6a2e (service), filesystem_pool: Some(ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External } } } }, OmicronZoneConfig { id: 41710ebd-a7df-4192-87ee-821f3687cee3 (service), filesystem_pool: Some(ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External } } } }, OmicronZoneConfig { id: 5b85d391-0238-40a9-b228-ddf2fb022360 (service), filesystem_pool: Some(ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External } } } }, OmicronZoneConfig { id: 4479233d-49d6-4267-9076-4d640d54f1e2 (service), filesystem_pool: Some(ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External } } } }, OmicronZoneConfig { id: 37b81eec-9fbb-4376-aec7-36daed3465bc (service), filesystem_pool: Some(ZpoolName { id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool), kind: External } } } }, OmicronZoneConfig { id: 7f79c18f-7b53-4b8a-86fc-bb23b923ea1d (service), filesystem_pool: Some(ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }), zone_type: ExternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External } }, http_address: [fd00:1122:3344:101::8]:5353, dns_address: 10.151.1.164:53, nic: NetworkInterface { id: 27d7c9ec-b8a8-4b31-ae12-c49ad36788f1, kind: Service { id: 7f79c18f-7b53-4b8a-86fc-bb23b923ea1d }, name: Name("external-dns-7f79c18f-7b53-4b8a-86fc-bb23b923ea1d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 238, 20])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } } }, OmicronZoneConfig { id: f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b (service), filesystem_pool: Some(ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }), zone_type: ExternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External } }, http_address: [fd00:1122:3344:101::9]:5353, dns_address: 10.151.1.165:53, nic: NetworkInterface { id: 29b92946-7588-4b89-8147-322049e1f231, kind: Service { id: f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b }, name: Name("external-dns-f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 239, 77])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } } }, OmicronZoneConfig { id: 9325209d-81b8-49e4-be6a-cedff3ef78c1 (service), filesystem_pool: Some(ZpoolName { id: 010339bd-18fb-4f1c-9455-7caf9e67e152 (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::a]:12221, external_ip: 10.151.1.166, nic: NetworkInterface { id: 0cd6826d-84a0-4cd5-b7c6-32cb83627e6c, kind: Service { id: 9325209d-81b8-49e4-be6a-cedff3ef78c1 }, name: Name("nexus-9325209d-81b8-49e4-be6a-cedff3ef78c1"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 192, 104])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: d2bdfdcf-c904-463a-87e2-60b209fd743f (service), filesystem_pool: Some(ZpoolName { id: 39aca50d-85fd-4ecc-a443-45b121ccfb9a (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::b]:12221, external_ip: 10.151.1.167, nic: NetworkInterface { id: ec778b7a-3947-4776-98f2-be2ced817d66, kind: Service { id: d2bdfdcf-c904-463a-87e2-60b209fd743f }, name: Name("nexus-d2bdfdcf-c904-463a-87e2-60b209fd743f"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 186, 218])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: 060b7a36-3ed1-4e47-89cf-3f46bfb09b6c (service), filesystem_pool: Some(ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::c]:12221, external_ip: 10.151.1.168, nic: NetworkInterface { id: f830fe44-0004-4968-88e2-304301a5a810, kind: Service { id: 060b7a36-3ed1-4e47-89cf-3f46bfb09b6c }, name: Name("nexus-060b7a36-3ed1-4e47-89cf-3f46bfb09b6c"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 157, 118])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: 3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0 (service), filesystem_pool: Some(ZpoolName { id: 39aca50d-85fd-4ecc-a443-45b121ccfb9a (zpool), kind: External }), zone_type: Oximeter { address: [fd00:1122:3344:101::d]:12223 } }, OmicronZoneConfig { id: f2d1f946-a521-4b2f-ab66-945d3b5d0ac5 (service), filesystem_pool: Some(ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }), zone_type: Clickhouse { address: [fd00:1122:3344:101::e]:8123, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External } } } }, OmicronZoneConfig { id: 0ac842d5-c5fe-4985-b91a-beeb82623eb1 (service), filesystem_pool: Some(ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::f]:17000 } }, OmicronZoneConfig { id: d792d4a3-6908-4d88-bf95-5371dd31e9d8 (service), filesystem_pool: Some(ZpoolName { id: 6b0637e8-562e-46b1-92a8-a7478ef38477 (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::10]:17000 } }, OmicronZoneConfig { id: c2e0eec3-76f5-4824-aa40-15149bda9204 (service), filesystem_pool: Some(ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::11]:17000 } }, OmicronZoneConfig { id: 9a8e4d7b-b87b-44df-95d9-4efe041c611f (service), filesystem_pool: Some(ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::12]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External } } } }, OmicronZoneConfig { id: b214f328-1474-4895-bf12-1b9492fb7338 (service), filesystem_pool: Some(ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::13]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External } } } }, OmicronZoneConfig { id: 16ad95ab-8ec8-4145-a12e-90482aa77a7f (service), filesystem_pool: Some(ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::14]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External } } } }, OmicronZoneConfig { id: 8d16b3eb-bad3-42c3-aace-86cdfefc9878 (service), filesystem_pool: Some(ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::15]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External } } } }, OmicronZoneConfig { id: 6b885e4b-9eef-45af-9061-b78cac8479f0 (service), filesystem_pool: Some(ZpoolName { id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::16]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool), kind: External } } } }, OmicronZoneConfig { id: 8bad491f-6df6-49cb-9ad0-7043316ba40c (service), filesystem_pool: Some(ZpoolName { id: 39aca50d-85fd-4ecc-a443-45b121ccfb9a (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::17]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 39aca50d-85fd-4ecc-a443-45b121ccfb9a (zpool), kind: External } } } }, OmicronZoneConfig { id: 33738c1e-05b4-47e5-9843-07c78c3a10c6 (service), filesystem_pool: Some(ZpoolName { id: 6b0637e8-562e-46b1-92a8-a7478ef38477 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::18]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 6b0637e8-562e-46b1-92a8-a7478ef38477 (zpool), kind: External } } } }, OmicronZoneConfig { id: 325eee21-9fb8-4009-ab47-7f3e7e0fa723 (service), filesystem_pool: Some(ZpoolName { id: 010339bd-18fb-4f1c-9455-7caf9e67e152 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::19]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 010339bd-18fb-4f1c-9455-7caf9e67e152 (zpool), kind: External } } } }, OmicronZoneConfig { id: ef8a820a-84e9-4141-b6bc-002a3603a655 (service), filesystem_pool: Some(ZpoolName { id: e3d9cf44-b37a-4529-b1ca-03b2819010d3 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::1a]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: e3d9cf44-b37a-4529-b1ca-03b2819010d3 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:459 sled_address = [fd00:1122:3344:101::1]:12345
10312024-11-02T03:56:28.409ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:49646
10322024-11-02T03:56:28.409ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2c96eacb-1841-4277-9578-6b84caeef9c6, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
10332024-11-02T03:56:28.409ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 2c96eacb-1841-4277-9578-6b84caeef9c6, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: InternalDns }, responder: ... }
10342024-11-02T03:56:28.439ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 1a7f019f-e208-4fae-b98e-fe6120f9b5e2, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
10352024-11-02T03:56:28.439ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 1a7f019f-e208-4fae-b98e-fe6120f9b5e2, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: InternalDns }, responder: ... }
10362024-11-02T03:56:28.462ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 0c3f42f2-ce05-428f-a91a-f78ae70ccccf, dataset_name: DatasetName { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
10372024-11-02T03:56:28.462ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 0c3f42f2-ce05-428f-a91a-f78ae70ccccf, dataset_name: DatasetName { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }, kind: InternalDns }, responder: ... }
10382024-11-02T03:56:28.483ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ea8c4184-9256-4357-bb1c-db39d54e6a2e, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
10392024-11-02T03:56:28.484ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: ea8c4184-9256-4357-bb1c-db39d54e6a2e, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: Cockroach }, responder: ... }
10402024-11-02T03:56:28.504ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 41710ebd-a7df-4192-87ee-821f3687cee3, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
10412024-11-02T03:56:28.504ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 41710ebd-a7df-4192-87ee-821f3687cee3, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10422024-11-02T03:56:28.524ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 5b85d391-0238-40a9-b228-ddf2fb022360, dataset_name: DatasetName { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
10432024-11-02T03:56:28.524ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 5b85d391-0238-40a9-b228-ddf2fb022360, dataset_name: DatasetName { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10442024-11-02T03:56:28.544ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 4479233d-49d6-4267-9076-4d640d54f1e2, dataset_name: DatasetName { pool_name: ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
10452024-11-02T03:56:28.544ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 4479233d-49d6-4267-9076-4d640d54f1e2, dataset_name: DatasetName { pool_name: ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10462024-11-02T03:56:28.565ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 37b81eec-9fbb-4376-aec7-36daed3465bc, dataset_name: DatasetName { pool_name: ZpoolName { id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:451
10472024-11-02T03:56:28.565ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 37b81eec-9fbb-4376-aec7-36daed3465bc, dataset_name: DatasetName { pool_name: ZpoolName { id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10482024-11-02T03:56:28.584ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 7f79c18f-7b53-4b8a-86fc-bb23b923ea1d, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: ExternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
10492024-11-02T03:56:28.584ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 7f79c18f-7b53-4b8a-86fc-bb23b923ea1d, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: ExternalDns }, responder: ... }
10502024-11-02T03:56:28.660ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: ExternalDns }, responder: ... }) file = sled-storage/src/manager.rs:451
10512024-11-02T03:56:28.660ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: ExternalDns }, responder: ... }
10522024-11-02T03:56:28.734ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f2d1f946-a521-4b2f-ab66-945d3b5d0ac5, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: Clickhouse }, responder: ... }) file = sled-storage/src/manager.rs:451
10532024-11-02T03:56:28.734ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: f2d1f946-a521-4b2f-ab66-945d3b5d0ac5, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: Clickhouse }, responder: ... }
10542024-11-02T03:56:28.811ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 9a8e4d7b-b87b-44df-95d9-4efe041c611f, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:451
10552024-11-02T03:56:28.811ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 9a8e4d7b-b87b-44df-95d9-4efe041c611f, dataset_name: DatasetName { pool_name: ZpoolName { id: 714116db-1daa-4c0c-b42f-cb5b471285ca (zpool), kind: External }, kind: Crucible }, responder: ... }
10562024-11-02T03:56:28.889ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: b214f328-1474-4895-bf12-1b9492fb7338, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:451
10572024-11-02T03:56:28.889ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: b214f328-1474-4895-bf12-1b9492fb7338, dataset_name: DatasetName { pool_name: ZpoolName { id: 68ab7f9e-24f1-481a-b99f-af893cdd74a3 (zpool), kind: External }, kind: Crucible }, responder: ... }
10582024-11-02T03:56:28.977ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 16ad95ab-8ec8-4145-a12e-90482aa77a7f, dataset_name: DatasetName { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:451
10592024-11-02T03:56:28.977ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 16ad95ab-8ec8-4145-a12e-90482aa77a7f, dataset_name: DatasetName { pool_name: ZpoolName { id: 4846b5cc-cb2b-4eca-94f1-57e5f623f3d0 (zpool), kind: External }, kind: Crucible }, responder: ... }
10602024-11-02T03:56:29.056ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 8d16b3eb-bad3-42c3-aace-86cdfefc9878, dataset_name: DatasetName { pool_name: ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:451
10612024-11-02T03:56:29.056ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 8d16b3eb-bad3-42c3-aace-86cdfefc9878, dataset_name: DatasetName { pool_name: ZpoolName { id: 61fda18b-b0fc-46cf-91c9-706f41acfc67 (zpool), kind: External }, kind: Crucible }, responder: ... }
10622024-11-02T03:56:29.131ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 6b885e4b-9eef-45af-9061-b78cac8479f0, dataset_name: DatasetName { pool_name: ZpoolName { id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:451
10632024-11-02T03:56:29.131ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 6b885e4b-9eef-45af-9061-b78cac8479f0, dataset_name: DatasetName { pool_name: ZpoolName { id: 09ce30cf-7836-438d-8cfe-4980ee894ac4 (zpool), kind: External }, kind: Crucible }, responder: ... }
10642024-11-02T03:56:29.213ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 8bad491f-6df6-49cb-9ad0-7043316ba40c, dataset_name: DatasetName { pool_name: ZpoolName { id: 39aca50d-85fd-4ecc-a443-45b121ccfb9a (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:451
10652024-11-02T03:56:29.213ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 8bad491f-6df6-49cb-9ad0-7043316ba40c, dataset_name: DatasetName { pool_name: ZpoolName { id: 39aca50d-85fd-4ecc-a443-45b121ccfb9a (zpool), kind: External }, kind: Crucible }, responder: ... }
10662024-11-02T03:56:29.289ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 33738c1e-05b4-47e5-9843-07c78c3a10c6, dataset_name: DatasetName { pool_name: ZpoolName { id: 6b0637e8-562e-46b1-92a8-a7478ef38477 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:451
10672024-11-02T03:56:29.289ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 33738c1e-05b4-47e5-9843-07c78c3a10c6, dataset_name: DatasetName { pool_name: ZpoolName { id: 6b0637e8-562e-46b1-92a8-a7478ef38477 (zpool), kind: External }, kind: Crucible }, responder: ... }
10682024-11-02T03:56:29.361ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 325eee21-9fb8-4009-ab47-7f3e7e0fa723, dataset_name: DatasetName { pool_name: ZpoolName { id: 010339bd-18fb-4f1c-9455-7caf9e67e152 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:451
10692024-11-02T03:56:29.361ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: 325eee21-9fb8-4009-ab47-7f3e7e0fa723, dataset_name: DatasetName { pool_name: ZpoolName { id: 010339bd-18fb-4f1c-9455-7caf9e67e152 (zpool), kind: External }, kind: Crucible }, responder: ... }
10702024-11-02T03:56:29.423ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ef8a820a-84e9-4141-b6bc-002a3603a655, dataset_name: DatasetName { pool_name: ZpoolName { id: e3d9cf44-b37a-4529-b1ca-03b2819010d3 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:451
10712024-11-02T03:56:29.423ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1059 request = NewFilesystemRequest { dataset_id: ef8a820a-84e9-4141-b6bc-002a3603a655, dataset_name: DatasetName { pool_name: ZpoolName { id: e3d9cf44-b37a-4529-b1ca-03b2819010d3 (zpool), kind: External }, kind: Crucible }, responder: ... }
10722024-11-02T03:56:29.488ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
10732024-11-02T03:56:29.488ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/6308f1ec-c08e-4282-a1c2-db37d6b8a044/config/omicron-zones.json file = common/src/ledger.rs:177
10742024-11-02T03:56:29.488ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/fda73319-9c6d-44d8-9377-b69537374c59/config/omicron-zones.json file = common/src/ledger.rs:177
10752024-11-02T03:56:29.488ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
10762024-11-02T03:56:29.494ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:41238
10772024-11-02T03:56:29.495ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 82 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:41238 req_id = ea332af5-a68d-4c11-9a0d-36559dafab56 response_code = 200 uri = /baseboard
10782024-11-02T03:56:29.657ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
10792024-11-02T03:56:29.657ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
10802024-11-02T03:56:29.658ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
10812024-11-02T03:56:29.658ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
10822024-11-02T03:56:29.658ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
10832024-11-02T03:56:29.658ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
10842024-11-02T03:56:29.658ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
10852024-11-02T03:56:29.669ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
10862024-11-02T03:56:29.671ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
10872024-11-02T03:56:29.693ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
10882024-11-02T03:56:29.843ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
10892024-11-02T03:56:29.843ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
10902024-11-02T03:56:29.843ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
10912024-11-02T03:56:29.843ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
10922024-11-02T03:56:29.872ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
10932024-11-02T03:56:29.872ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
10942024-11-02T03:56:29.872ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
10952024-11-02T03:56:29.872ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
10962024-11-02T03:56:29.872ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
10972024-11-02T03:56:29.872ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338 file = illumos-utils/src/zone.rs:325
10982024-11-02T03:56:29.877ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_8d16b3eb-bad3-42c3-aace-86cdfefc9878 file = illumos-utils/src/zone.rs:325
10992024-11-02T03:56:29.884ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8 file = illumos-utils/src/zone.rs:325
11002024-11-02T03:56:29.889ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_6b885e4b-9eef-45af-9061-b78cac8479f0 file = illumos-utils/src/zone.rs:325
11012024-11-02T03:56:29.895ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_8bad491f-6df6-49cb-9ad0-7043316ba40c file = illumos-utils/src/zone.rs:325
11022024-11-02T03:56:29.900ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6 file = illumos-utils/src/zone.rs:325
11032024-11-02T03:56:29.906ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f file = illumos-utils/src/zone.rs:325
11042024-11-02T03:56:30.053ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338 file = illumos-utils/src/zone.rs:363
11052024-11-02T03:56:30.057ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_8d16b3eb-bad3-42c3-aace-86cdfefc9878 file = illumos-utils/src/zone.rs:363
11062024-11-02T03:56:30.062ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8 file = illumos-utils/src/zone.rs:363
11072024-11-02T03:56:30.068ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_6b885e4b-9eef-45af-9061-b78cac8479f0 file = illumos-utils/src/zone.rs:363
11082024-11-02T03:56:30.072ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_8bad491f-6df6-49cb-9ad0-7043316ba40c file = illumos-utils/src/zone.rs:363
11092024-11-02T03:56:30.077ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6 file = illumos-utils/src/zone.rs:363
11102024-11-02T03:56:30.083ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f file = illumos-utils/src/zone.rs:363
11112024-11-02T03:56:30.087ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_16ad95ab-8ec8-4145-a12e-90482aa77a7f file = illumos-utils/src/zone.rs:325
11122024-11-02T03:56:30.092ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0 file = illumos-utils/src/zone.rs:325
11132024-11-02T03:56:30.097ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655 file = illumos-utils/src/zone.rs:325
11142024-11-02T03:56:30.103ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_clickhouse_f2d1f946-a521-4b2f-ab66-945d3b5d0ac5 file = illumos-utils/src/zone.rs:325
11152024-11-02T03:56:30.108ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723 file = illumos-utils/src/zone.rs:325
11162024-11-02T03:56:30.115ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1 file = illumos-utils/src/zone.rs:325
11172024-11-02T03:56:30.121ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_c2e0eec3-76f5-4824-aa40-15149bda9204 file = illumos-utils/src/zone.rs:325
11182024-11-02T03:56:30.128ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_16ad95ab-8ec8-4145-a12e-90482aa77a7f file = illumos-utils/src/zone.rs:363
11192024-11-02T03:56:30.134ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0 file = illumos-utils/src/zone.rs:363
11202024-11-02T03:56:30.139ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655 file = illumos-utils/src/zone.rs:363
11212024-11-02T03:56:30.144ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_clickhouse_f2d1f946-a521-4b2f-ab66-945d3b5d0ac5 file = illumos-utils/src/zone.rs:363
11222024-11-02T03:56:30.149ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723 file = illumos-utils/src/zone.rs:363
11232024-11-02T03:56:30.156ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1 file = illumos-utils/src/zone.rs:363
11242024-11-02T03:56:30.162ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_c2e0eec3-76f5-4824-aa40-15149bda9204 file = illumos-utils/src/zone.rs:363
11252024-11-02T03:56:30.228ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
11262024-11-02T03:56:30.284ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
11272024-11-02T03:56:30.338ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
11282024-11-02T03:56:30.394ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
11292024-11-02T03:56:30.467ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
11302024-11-02T03:56:30.470ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
11312024-11-02T03:56:30.470ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
11322024-11-02T03:56:30.537ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:430 port = Port { inner: PortInner(PortData { name: "opte1", ip: 172.30.2.7, mac: MacAddr6([168, 64, 37, 255, 157, 118]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
11332024-11-02T03:56:30.538ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1260 zone_type = nexus
11342024-11-02T03:56:30.540ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
11352024-11-02T03:56:30.540ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
11362024-11-02T03:56:30.615ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:430 port = Port { inner: PortInner(PortData { name: "opte2", ip: 172.30.1.6, mac: MacAddr6([168, 64, 37, 255, 239, 77]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.1.1 } }) }
11372024-11-02T03:56:30.615ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1260 zone_type = external_dns
11382024-11-02T03:56:30.617ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
11392024-11-02T03:56:30.617ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
11402024-11-02T03:56:30.674ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:430 port = Port { inner: PortInner(PortData { name: "opte3", ip: 172.30.1.5, mac: MacAddr6([168, 64, 37, 255, 238, 20]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.1.1 } }) }
11412024-11-02T03:56:30.674ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1260 zone_type = external_dns
11422024-11-02T03:56:30.675ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
11432024-11-02T03:56:30.676ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
11442024-11-02T03:56:30.733ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:430 port = Port { inner: PortInner(PortData { name: "opte4", ip: 172.30.2.5, mac: MacAddr6([168, 64, 37, 255, 192, 104]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
11452024-11-02T03:56:30.733ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1260 zone_type = nexus
11462024-11-02T03:56:30.734ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
11472024-11-02T03:56:30.734ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
11482024-11-02T03:56:30.786ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:430 port = Port { inner: PortInner(PortData { name: "opte5", ip: 172.30.2.6, mac: MacAddr6([168, 64, 37, 255, 186, 218]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
11492024-11-02T03:56:30.786ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1260 zone_type = nexus
11502024-11-02T03:56:30.787ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.1.167 file = clients/dpd-client/src/lib.rs:83
11512024-11-02T03:56:30.787ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.1.167 file = clients/dpd-client/src/lib.rs:90
11522024-11-02T03:56:30.787ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.1.165 file = clients/dpd-client/src/lib.rs:83
11532024-11-02T03:56:30.787ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.1.165 file = clients/dpd-client/src/lib.rs:90
11542024-11-02T03:56:30.787ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.1.164 file = clients/dpd-client/src/lib.rs:83
11552024-11-02T03:56:30.787ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.1.164 file = clients/dpd-client/src/lib.rs:90
11562024-11-02T03:56:30.787ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.1.166 file = clients/dpd-client/src/lib.rs:83
11572024-11-02T03:56:30.787ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.1.166 file = clients/dpd-client/src/lib.rs:90
11582024-11-02T03:56:30.787ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.1.168 file = clients/dpd-client/src/lib.rs:83
11592024-11-02T03:56:30.787ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.1.168 file = clients/dpd-client/src/lib.rs:90
11602024-11-02T03:56:30.788ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.1.167 file = clients/dpd-client/src/lib.rs:118
11612024-11-02T03:56:30.790ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
11622024-11-02T03:56:30.827ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.1.168 file = clients/dpd-client/src/lib.rs:118
11632024-11-02T03:56:30.830ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
11642024-11-02T03:56:30.830ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.1.165 file = clients/dpd-client/src/lib.rs:118
11652024-11-02T03:56:30.834ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.1.164 file = clients/dpd-client/src/lib.rs:118
11662024-11-02T03:56:30.838ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.1.166 file = clients/dpd-client/src/lib.rs:118
11672024-11-02T03:56:30.838ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
11682024-11-02T03:56:30.838ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
11692024-11-02T03:56:30.882ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
11702024-11-02T03:56:30.993ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_d2bdfdcf-c904-463a-87e2-60b209fd743f file = illumos-utils/src/zone.rs:325
11712024-11-02T03:56:31.010ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_060b7a36-3ed1-4e47-89cf-3f46bfb09b6c file = illumos-utils/src/zone.rs:325
11722024-11-02T03:56:31.044ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_external_dns_f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b file = illumos-utils/src/zone.rs:325
11732024-11-02T03:56:31.062ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_external_dns_7f79c18f-7b53-4b8a-86fc-bb23b923ea1d file = illumos-utils/src/zone.rs:325
11742024-11-02T03:56:31.089ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_9325209d-81b8-49e4-be6a-cedff3ef78c1 file = illumos-utils/src/zone.rs:325
11752024-11-02T03:56:31.110ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_d2bdfdcf-c904-463a-87e2-60b209fd743f file = illumos-utils/src/zone.rs:363
11762024-11-02T03:56:31.137ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_060b7a36-3ed1-4e47-89cf-3f46bfb09b6c file = illumos-utils/src/zone.rs:363
11772024-11-02T03:56:31.164ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_external_dns_f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b file = illumos-utils/src/zone.rs:363
11782024-11-02T03:56:31.180ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_external_dns_7f79c18f-7b53-4b8a-86fc-bb23b923ea1d file = illumos-utils/src/zone.rs:363
11792024-11-02T03:56:31.195ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_9325209d-81b8-49e4-be6a-cedff3ef78c1 file = illumos-utils/src/zone.rs:363
11802024-11-02T03:56:37.266ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_16ad95ab-8ec8-4145-a12e-90482aa77a7f: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService17'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::14'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_4846b5cc-cb2b-4eca-94f1-57e5f623f3d0/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::14'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='c42b4e66-28b3-40c6-80a4-953a07c6b774'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11812024-11-02T03:56:37.290ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_6b885e4b-9eef-45af-9061-b78cac8479f0: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService13'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::16'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_09ce30cf-7836-438d-8cfe-4980ee894ac4/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::16'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='4b9a8dea-97ef-476f-934c-0ff31896dce3'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11822024-11-02T03:56:37.296ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService15'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::18'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_6b0637e8-562e-46b1-92a8-a7478ef38477/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::18'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='c07baa99-7c0e-4fed-bd1b-8cb00d23bdf0'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11832024-11-02T03:56:37.298ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService16'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::12'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_714116db-1daa-4c0c-b42f-cb5b471285ca/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::12'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='f71e4cfd-beaa-4656-8e76-17203242a004'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11842024-11-02T03:56:37.298ZINFOSledAgent (ServiceManager): Booting oxz_crucible_16ad95ab-8ec8-4145-a12e-90482aa77a7f zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_16ad95ab-8ec8-4145-a12e-90482aa77a7f
11852024-11-02T03:56:37.927ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService21'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::19'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_010339bd-18fb-4f1c-9455-7caf9e67e152/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::19'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='9e2f6aac-e4ec-4eca-b224-62595c36fb18'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11862024-11-02T03:56:37.946ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService10'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::13'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_68ab7f9e-24f1-481a-b99f-af893cdd74a3/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::13'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='6adecc3b-fbf9-4442-b020-86dc769d3bd8'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11872024-11-02T03:56:37.954ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_8d16b3eb-bad3-42c3-aace-86cdfefc9878: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService11'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::15'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_61fda18b-b0fc-46cf-91c9-706f41acfc67/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::15'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='7d745829-eb31-4a86-bd9e-0e615b9a27fe'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11882024-11-02T03:56:37.966ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService12'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::10'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::10'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11892024-11-02T03:56:37.970ZINFOSledAgent (ServiceManager): Booting oxz_crucible_6b885e4b-9eef-45af-9061-b78cac8479f0 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_6b885e4b-9eef-45af-9061-b78cac8479f0
11902024-11-02T03:56:38.135ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_8bad491f-6df6-49cb-9ad0-7043316ba40c: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService14'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::17'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_39aca50d-85fd-4ecc-a443-45b121ccfb9a/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::17'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='5c476a5d-76f5-4d46-a8dc-d75cf79ca246'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11912024-11-02T03:56:38.135ZINFOSledAgent (ServiceManager): Booting oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6
11922024-11-02T03:56:38.163ZINFOSledAgent (ServiceManager): Booting oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f
11932024-11-02T03:56:38.189ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService19'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::1a'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_e3d9cf44-b37a-4529-b1ca-03b2819010d3/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::1a'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='6186a726-4e8f-48f1-99d9-99d6dc1a9f14'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11942024-11-02T03:56:38.193ZINFOSledAgent (ServiceManager): Booting oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723
11952024-11-02T03:56:38.217ZINFOSledAgent (ServiceManager): Booting oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338
11962024-11-02T03:56:38.271ZINFOSledAgent (ServiceManager): Booting oxz_crucible_8d16b3eb-bad3-42c3-aace-86cdfefc9878 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_8d16b3eb-bad3-42c3-aace-86cdfefc9878
11972024-11-02T03:56:38.292ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8
11982024-11-02T03:56:38.725ZINFOSledAgent (ServiceManager): Booting oxz_crucible_8bad491f-6df6-49cb-9ad0-7043316ba40c zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_8bad491f-6df6-49cb-9ad0-7043316ba40c
11992024-11-02T03:56:40.849ZINFOSledAgent (ServiceManager): Booting oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655
12002024-11-02T03:56:41.519ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService22'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::f'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::f'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
12012024-11-02T03:56:41.528ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1
12022024-11-02T03:56:41.940ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338") failed: Property not found. retry in 37.578021ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338
12032024-11-02T03:56:41.951ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8d16b3eb-bad3-42c3-aace-86cdfefc9878") failed: Property not found. retry in 54.497614ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8d16b3eb-bad3-42c3-aace-86cdfefc9878
12042024-11-02T03:56:41.961ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8") failed: Property not found. retry in 54.917323ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8
12052024-11-02T03:56:41.973ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6b885e4b-9eef-45af-9061-b78cac8479f0") failed: Property not found. retry in 37.108019ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6b885e4b-9eef-45af-9061-b78cac8479f0
12062024-11-02T03:56:41.984ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8bad491f-6df6-49cb-9ad0-7043316ba40c") failed: Property not found. retry in 67.973391ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8bad491f-6df6-49cb-9ad0-7043316ba40c
12072024-11-02T03:56:41.995ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6") failed: Property not found. retry in 68.506318ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6
12082024-11-02T03:56:42.008ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f") failed: Property not found. retry in 69.010144ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f
12092024-11-02T03:56:42.021ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16ad95ab-8ec8-4145-a12e-90482aa77a7f") failed: Property not found. retry in 64.726456ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16ad95ab-8ec8-4145-a12e-90482aa77a7f
12102024-11-02T03:56:42.033ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655") failed: Property not found. retry in 62.402749ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655
12112024-11-02T03:56:42.047ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723") failed: Property not found. retry in 34.905817ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723
12122024-11-02T03:56:42.061ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1") failed: Property not found. retry in 32.249741ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1
12132024-11-02T03:56:42.074ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338") failed: Property not found. retry in 78.163451ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338
12142024-11-02T03:56:42.085ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8d16b3eb-bad3-42c3-aace-86cdfefc9878") failed: Property not found. retry in 115.361842ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8d16b3eb-bad3-42c3-aace-86cdfefc9878
12152024-11-02T03:56:42.099ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8") failed: Property not found. retry in 93.226943ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8
12162024-11-02T03:56:42.110ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6b885e4b-9eef-45af-9061-b78cac8479f0") failed: Property not found. retry in 86.355939ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6b885e4b-9eef-45af-9061-b78cac8479f0
12172024-11-02T03:56:42.120ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8bad491f-6df6-49cb-9ad0-7043316ba40c") failed: Property not found. retry in 85.963161ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8bad491f-6df6-49cb-9ad0-7043316ba40c
12182024-11-02T03:56:42.131ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6") failed: Property not found. retry in 146.708195ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6
12192024-11-02T03:56:42.141ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f") failed: Property not found. retry in 85.723642ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f
12202024-11-02T03:56:42.150ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16ad95ab-8ec8-4145-a12e-90482aa77a7f") failed: Property not found. retry in 97.986611ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16ad95ab-8ec8-4145-a12e-90482aa77a7f
12212024-11-02T03:56:42.170ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655") failed: Property not found. retry in 62.84848ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655
12222024-11-02T03:56:42.183ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723") failed: Property not found. retry in 136.551407ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723
12232024-11-02T03:56:42.199ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1") failed: Property not found. retry in 130.395765ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1
12242024-11-02T03:56:42.212ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338") failed: Property not found. retry in 298.960019ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338
12252024-11-02T03:56:42.226ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8d16b3eb-bad3-42c3-aace-86cdfefc9878") failed: Property not found. retry in 169.324865ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8d16b3eb-bad3-42c3-aace-86cdfefc9878
12262024-11-02T03:56:42.241ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8") failed: Property not found. retry in 273.523415ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8
12272024-11-02T03:56:42.256ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6b885e4b-9eef-45af-9061-b78cac8479f0") failed: Property not found. retry in 255.183501ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6b885e4b-9eef-45af-9061-b78cac8479f0
12282024-11-02T03:56:42.272ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8bad491f-6df6-49cb-9ad0-7043316ba40c") failed: Property not found. retry in 271.627723ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8bad491f-6df6-49cb-9ad0-7043316ba40c
12292024-11-02T03:56:42.286ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f") failed: Property not found. retry in 169.356845ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f
12302024-11-02T03:56:42.302ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16ad95ab-8ec8-4145-a12e-90482aa77a7f") failed: Property not found. retry in 117.690928ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16ad95ab-8ec8-4145-a12e-90482aa77a7f
12312024-11-02T03:56:42.319ZINFOSledAgent (ServiceManager): Profile for oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService18'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::d'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/oximeter"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="id" value='3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0'/> <propval type="astring" name="address" value='[fd00:1122:3344:101::d]:12223'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
12322024-11-02T03:56:42.336ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655") failed: Property not found. retry in 104.598575ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655
12332024-11-02T03:56:42.355ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723") failed: Property not found. retry in 234.11881ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723
12342024-11-02T03:56:42.368ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1") failed: Property not found. retry in 102.691196ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1
12352024-11-02T03:56:42.382ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6") failed: Property not found. retry in 298.733441ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6
12362024-11-02T03:56:42.382ZINFOSledAgent (ServiceManager): Booting oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0
12372024-11-02T03:56:42.409ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8d16b3eb-bad3-42c3-aace-86cdfefc9878") failed: Property not found. retry in 516.772257ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8d16b3eb-bad3-42c3-aace-86cdfefc9878
12382024-11-02T03:56:42.437ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16ad95ab-8ec8-4145-a12e-90482aa77a7f") failed: Property not found. retry in 397.996365ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16ad95ab-8ec8-4145-a12e-90482aa77a7f
12392024-11-02T03:56:42.456ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655") failed: Property not found. retry in 439.118932ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655
12402024-11-02T03:56:42.470ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f") failed: Property not found. retry in 498.238324ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f
12412024-11-02T03:56:42.484ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1") failed: Property not found. retry in 212.393481ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1
12422024-11-02T03:56:42.525ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338") failed: Property not found. retry in 311.853117ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338
12432024-11-02T03:56:42.538ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6b885e4b-9eef-45af-9061-b78cac8479f0") failed: Property not found. retry in 391.332029ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6b885e4b-9eef-45af-9061-b78cac8479f0
12442024-11-02T03:56:42.551ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8") failed: Property not found. retry in 591.95872ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8
12452024-11-02T03:56:42.565ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8bad491f-6df6-49cb-9ad0-7043316ba40c") failed: Property not found. retry in 508.272019ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8bad491f-6df6-49cb-9ad0-7043316ba40c
12462024-11-02T03:56:42.602ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723") failed: Property not found. retry in 468.232053ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723
12472024-11-02T03:56:42.629ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0") failed: Property not found. retry in 70.821679ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0
12482024-11-02T03:56:42.695ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6") failed: Property not found. retry in 375.41519ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6
12492024-11-02T03:56:42.706ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1") failed: Property not found. retry in 623.590919ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1
12502024-11-02T03:56:42.715ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0") failed: Property not found. retry in 108.047408ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0
12512024-11-02T03:56:42.836ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0") failed: Property not found. retry in 168.214991ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0
12522024-11-02T03:56:42.846ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16ad95ab-8ec8-4145-a12e-90482aa77a7f") failed: Property not found. retry in 968.538359ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16ad95ab-8ec8-4145-a12e-90482aa77a7f
12532024-11-02T03:56:42.856ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338") failed: Property not found. retry in 961.391437ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338
12542024-11-02T03:56:42.909ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655") failed: Property not found. retry in 797.411472ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655
12552024-11-02T03:56:42.940ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8d16b3eb-bad3-42c3-aace-86cdfefc9878") failed: Property not found. retry in 609.183542ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8d16b3eb-bad3-42c3-aace-86cdfefc9878
12562024-11-02T03:56:42.953ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6b885e4b-9eef-45af-9061-b78cac8479f0") failed: Property not found. retry in 1.035455746s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6b885e4b-9eef-45af-9061-b78cac8479f0
12572024-11-02T03:56:42.988ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f") failed: Property not found. retry in 598.281215ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f
12582024-11-02T03:56:43.019ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0") failed: Property not found. retry in 545.547731ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0
12592024-11-02T03:56:43.087ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6") failed: Property not found. retry in 1.184130473s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6
12602024-11-02T03:56:43.102ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723") failed: Property not found. retry in 695.285811ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723
12612024-11-02T03:56:43.116ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8bad491f-6df6-49cb-9ad0-7043316ba40c") failed: Property not found. retry in 947.081143ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8bad491f-6df6-49cb-9ad0-7043316ba40c
12622024-11-02T03:56:43.155ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8") failed: Property not found. retry in 1.189172803s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8
12632024-11-02T03:56:43.581ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1") failed: Property not found. retry in 1.171186123s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1
12642024-11-02T03:56:43.629ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8d16b3eb-bad3-42c3-aace-86cdfefc9878") failed: Property not found. retry in 1.015387005s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8d16b3eb-bad3-42c3-aace-86cdfefc9878
12652024-11-02T03:56:43.668ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0") failed: Property not found. retry in 818.83697ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0
12662024-11-02T03:56:43.699ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f") failed: Property not found. retry in 1.353857363s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f
12672024-11-02T03:56:43.754ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655") failed: Property not found. retry in 994.364984ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655
12682024-11-02T03:56:43.754ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_c2e0eec3-76f5-4824-aa40-15149bda9204: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService23'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::11'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::11'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
12692024-11-02T03:56:43.754ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_c2e0eec3-76f5-4824-aa40-15149bda9204 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_c2e0eec3-76f5-4824-aa40-15149bda9204
12702024-11-02T03:56:43.835ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723") failed: Property not found. retry in 1.147582023s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723
12712024-11-02T03:56:43.883ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338") failed: Property not found. retry in 1.376327896s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338
12722024-11-02T03:56:43.922ZINFOSledAgent (ServiceManager): Profile for oxz_external_dns_f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService26'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::9'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte2'/> <propval type="astring" name="gateway" value='172.30.1.1'/> <propval type="astring" name="ip" value='172.30.1.6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/external_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::9]:5353'/> <propval type="astring" name="dns_address" value='172.30.1.6:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
12732024-11-02T03:56:43.940ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16ad95ab-8ec8-4145-a12e-90482aa77a7f") failed: Property not found. retry in 809.373692ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16ad95ab-8ec8-4145-a12e-90482aa77a7f
12742024-11-02T03:56:43.940ZINFOSledAgent (ServiceManager): Booting oxz_external_dns_f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_external_dns_f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b
12752024-11-02T03:56:44.017ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6b885e4b-9eef-45af-9061-b78cac8479f0") failed: Property not found. retry in 1.107743344s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6b885e4b-9eef-45af-9061-b78cac8479f0
12762024-11-02T03:56:44.162ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8bad491f-6df6-49cb-9ad0-7043316ba40c") failed: Property not found. retry in 1.016591739s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8bad491f-6df6-49cb-9ad0-7043316ba40c
12772024-11-02T03:56:44.235ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_c2e0eec3-76f5-4824-aa40-15149bda9204") failed: Property not found. retry in 58.382603ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_c2e0eec3-76f5-4824-aa40-15149bda9204
12782024-11-02T03:56:44.309ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6") failed: Property not found. retry in 1.188325341s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6
12792024-11-02T03:56:44.373ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 2.626493568s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:424 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
12802024-11-02T03:56:44.379ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b") failed: Property not found. retry in 55.019765ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b
12812024-11-02T03:56:44.394ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_c2e0eec3-76f5-4824-aa40-15149bda9204") failed: Property not found. retry in 132.234171ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_c2e0eec3-76f5-4824-aa40-15149bda9204
12822024-11-02T03:56:44.408ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8") failed: Property not found. retry in 570.945915ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8
12832024-11-02T03:56:44.447ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b") failed: Property not found. retry in 121.473184ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b
12842024-11-02T03:56:44.504ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0") failed: Property not found. retry in 663.974942ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0
12852024-11-02T03:56:44.539ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_c2e0eec3-76f5-4824-aa40-15149bda9204") failed: Property not found. retry in 269.786504ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_c2e0eec3-76f5-4824-aa40-15149bda9204
12862024-11-02T03:56:44.584ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b") failed: Property not found. retry in 174.324364ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b
12872024-11-02T03:56:44.621ZINFOSledAgent (ServiceManager): Profile for oxz_external_dns_7f79c18f-7b53-4b8a-86fc-bb23b923ea1d: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService27'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::8'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte3'/> <propval type="astring" name="gateway" value='172.30.1.1'/> <propval type="astring" name="ip" value='172.30.1.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/external_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::8]:5353'/> <propval type="astring" name="dns_address" value='172.30.1.5:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
12882024-11-02T03:56:44.622ZINFOSledAgent (ServiceManager): Booting oxz_external_dns_7f79c18f-7b53-4b8a-86fc-bb23b923ea1d zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_external_dns_7f79c18f-7b53-4b8a-86fc-bb23b923ea1d
12892024-11-02T03:56:44.658ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8d16b3eb-bad3-42c3-aace-86cdfefc9878") failed: Property not found. retry in 1.250716667s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8d16b3eb-bad3-42c3-aace-86cdfefc9878
12902024-11-02T03:56:44.763ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655") failed: Property not found. retry in 625.058544ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655
12912024-11-02T03:56:44.774ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b") failed: Property not found. retry in 297.119517ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b
12922024-11-02T03:56:44.786ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16ad95ab-8ec8-4145-a12e-90482aa77a7f") failed: Property not found. retry in 1.074486661s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16ad95ab-8ec8-4145-a12e-90482aa77a7f
12932024-11-02T03:56:44.798ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1") failed: Property not found. retry in 615.508602ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1
12942024-11-02T03:56:44.820ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_c2e0eec3-76f5-4824-aa40-15149bda9204") failed: Property not found. retry in 337.676463ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_c2e0eec3-76f5-4824-aa40-15149bda9204
12952024-11-02T03:56:44.898ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7f79c18f-7b53-4b8a-86fc-bb23b923ea1d") failed: Property not found. retry in 31.710599ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7f79c18f-7b53-4b8a-86fc-bb23b923ea1d
12962024-11-02T03:56:44.940ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7f79c18f-7b53-4b8a-86fc-bb23b923ea1d") failed: Property not found. retry in 71.674282ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7f79c18f-7b53-4b8a-86fc-bb23b923ea1d
12972024-11-02T03:56:44.989ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8") failed: Property not found. retry in 781.619214ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8
12982024-11-02T03:56:44.998ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723") failed: Property not found. retry in 1.116276039s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723
12992024-11-02T03:56:45.022ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7f79c18f-7b53-4b8a-86fc-bb23b923ea1d") failed: Property not found. retry in 197.785333ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7f79c18f-7b53-4b8a-86fc-bb23b923ea1d
13002024-11-02T03:56:45.064ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f") failed: Property not found. retry in 1.464517765s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f
13012024-11-02T03:56:45.082ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b") failed: Property not found. retry in 1.1882681s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b
13022024-11-02T03:56:45.136ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6b885e4b-9eef-45af-9061-b78cac8479f0") failed: Property not found. retry in 1.159965383s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6b885e4b-9eef-45af-9061-b78cac8479f0
13032024-11-02T03:56:45.182ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_c2e0eec3-76f5-4824-aa40-15149bda9204") failed: Property not found. retry in 508.95839ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_c2e0eec3-76f5-4824-aa40-15149bda9204
13042024-11-02T03:56:45.192ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8bad491f-6df6-49cb-9ad0-7043316ba40c") failed: Property not found. retry in 1.109640004s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8bad491f-6df6-49cb-9ad0-7043316ba40c
13052024-11-02T03:56:45.201ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0") failed: Property not found. retry in 1.041391527s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0
13062024-11-02T03:56:45.232ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7f79c18f-7b53-4b8a-86fc-bb23b923ea1d") failed: Property not found. retry in 323.309865ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7f79c18f-7b53-4b8a-86fc-bb23b923ea1d
13072024-11-02T03:56:45.272ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338") failed: Property not found. retry in 757.159532ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338
13082024-11-02T03:56:45.401ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655") failed: Property not found. retry in 1.34704903s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655
13092024-11-02T03:56:45.425ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1") failed: Property not found. retry in 662.012322ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1
13102024-11-02T03:56:45.445ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_060b7a36-3ed1-4e47-89cf-3f46bfb09b6c: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService25'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::c'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte1'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.7'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13112024-11-02T03:56:45.445ZINFOSledAgent (ServiceManager): Booting oxz_nexus_060b7a36-3ed1-4e47-89cf-3f46bfb09b6c zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_060b7a36-3ed1-4e47-89cf-3f46bfb09b6c
13122024-11-02T03:56:45.465ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_d2bdfdcf-c904-463a-87e2-60b209fd743f: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService24'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::b'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte5'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13132024-11-02T03:56:45.465ZINFOSledAgent (ServiceManager): Booting oxz_nexus_d2bdfdcf-c904-463a-87e2-60b209fd743f zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_d2bdfdcf-c904-463a-87e2-60b209fd743f
13142024-11-02T03:56:45.498ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_9325209d-81b8-49e4-be6a-cedff3ef78c1: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService28'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::a'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte4'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13152024-11-02T03:56:45.498ZINFOSledAgent (ServiceManager): Booting oxz_nexus_9325209d-81b8-49e4-be6a-cedff3ef78c1 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_9325209d-81b8-49e4-be6a-cedff3ef78c1
13162024-11-02T03:56:45.516ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6") failed: Property not found. retry in 1.327643867s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6
13172024-11-02T03:56:45.569ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7f79c18f-7b53-4b8a-86fc-bb23b923ea1d") failed: Property not found. retry in 825.963398ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7f79c18f-7b53-4b8a-86fc-bb23b923ea1d
13182024-11-02T03:56:45.653ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d2bdfdcf-c904-463a-87e2-60b209fd743f") failed: Property not found. retry in 40.48457ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d2bdfdcf-c904-463a-87e2-60b209fd743f
13192024-11-02T03:56:45.664ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_060b7a36-3ed1-4e47-89cf-3f46bfb09b6c") failed: Property not found. retry in 73.204295ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_060b7a36-3ed1-4e47-89cf-3f46bfb09b6c
13202024-11-02T03:56:45.703ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_c2e0eec3-76f5-4824-aa40-15149bda9204") failed: Property not found. retry in 595.037321ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_c2e0eec3-76f5-4824-aa40-15149bda9204
13212024-11-02T03:56:45.711ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_9325209d-81b8-49e4-be6a-cedff3ef78c1") failed: Property not found. retry in 44.842481ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_9325209d-81b8-49e4-be6a-cedff3ef78c1
13222024-11-02T03:56:45.719ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d2bdfdcf-c904-463a-87e2-60b209fd743f") failed: Property not found. retry in 136.037149ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d2bdfdcf-c904-463a-87e2-60b209fd743f
13232024-11-02T03:56:45.748ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_060b7a36-3ed1-4e47-89cf-3f46bfb09b6c") failed: Property not found. retry in 126.680904ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_060b7a36-3ed1-4e47-89cf-3f46bfb09b6c
13242024-11-02T03:56:45.767ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_9325209d-81b8-49e4-be6a-cedff3ef78c1") failed: Property not found. retry in 120.671777ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_9325209d-81b8-49e4-be6a-cedff3ef78c1
13252024-11-02T03:56:45.782ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8") failed: Property not found. retry in 1.094641625s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8
13262024-11-02T03:56:45.871ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d2bdfdcf-c904-463a-87e2-60b209fd743f") failed: Property not found. retry in 181.37528ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d2bdfdcf-c904-463a-87e2-60b209fd743f
13272024-11-02T03:56:45.882ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16ad95ab-8ec8-4145-a12e-90482aa77a7f") failed: Property not found. retry in 1.284034187s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16ad95ab-8ec8-4145-a12e-90482aa77a7f
13282024-11-02T03:56:45.893ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_060b7a36-3ed1-4e47-89cf-3f46bfb09b6c") failed: Property not found. retry in 277.974395ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_060b7a36-3ed1-4e47-89cf-3f46bfb09b6c
13292024-11-02T03:56:45.907ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_9325209d-81b8-49e4-be6a-cedff3ef78c1") failed: Property not found. retry in 119.118666ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_9325209d-81b8-49e4-be6a-cedff3ef78c1
13302024-11-02T03:56:45.931ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8d16b3eb-bad3-42c3-aace-86cdfefc9878") failed: Property not found. retry in 1.403952419s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8d16b3eb-bad3-42c3-aace-86cdfefc9878
13312024-11-02T03:56:46.040ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_9325209d-81b8-49e4-be6a-cedff3ef78c1") failed: Property not found. retry in 448.350557ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_9325209d-81b8-49e4-be6a-cedff3ef78c1
13322024-11-02T03:56:46.050ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338") failed: Property not found. retry in 912.217024ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338
13332024-11-02T03:56:46.063ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d2bdfdcf-c904-463a-87e2-60b209fd743f") failed: Property not found. retry in 433.343423ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d2bdfdcf-c904-463a-87e2-60b209fd743f
13342024-11-02T03:56:46.098ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1") failed: Property not found. retry in 581.527237ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1
13352024-11-02T03:56:46.126ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723") failed: Property not found. retry in 679.250438ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723
13362024-11-02T03:56:46.184ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_060b7a36-3ed1-4e47-89cf-3f46bfb09b6c") failed: Property not found. retry in 487.898336ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_060b7a36-3ed1-4e47-89cf-3f46bfb09b6c
13372024-11-02T03:56:46.257ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0") failed: Property not found. retry in 1.122973289s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0
13382024-11-02T03:56:46.284ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b") failed: Property not found. retry in 808.554604ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b
13392024-11-02T03:56:46.309ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6b885e4b-9eef-45af-9061-b78cac8479f0") failed: Property not found. retry in 1.330958664s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6b885e4b-9eef-45af-9061-b78cac8479f0
13402024-11-02T03:56:46.321ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8bad491f-6df6-49cb-9ad0-7043316ba40c") failed: Property not found. retry in 1.0102112s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8bad491f-6df6-49cb-9ad0-7043316ba40c
13412024-11-02T03:56:46.332ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_c2e0eec3-76f5-4824-aa40-15149bda9204") failed: Property not found. retry in 650.671743ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_c2e0eec3-76f5-4824-aa40-15149bda9204
13422024-11-02T03:56:46.409ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7f79c18f-7b53-4b8a-86fc-bb23b923ea1d") failed: Property not found. retry in 532.39685ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7f79c18f-7b53-4b8a-86fc-bb23b923ea1d
13432024-11-02T03:56:46.501ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_9325209d-81b8-49e4-be6a-cedff3ef78c1") failed: Property not found. retry in 610.144471ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_9325209d-81b8-49e4-be6a-cedff3ef78c1
13442024-11-02T03:56:46.514ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d2bdfdcf-c904-463a-87e2-60b209fd743f") failed: Property not found. retry in 732.011463ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d2bdfdcf-c904-463a-87e2-60b209fd743f
13452024-11-02T03:56:46.539ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f") failed: Property not found. retry in 1.09298787s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f
13462024-11-02T03:56:46.685ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_060b7a36-3ed1-4e47-89cf-3f46bfb09b6c") failed: Property not found. retry in 523.099448ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_060b7a36-3ed1-4e47-89cf-3f46bfb09b6c
13472024-11-02T03:56:46.696ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1") failed: Property not found. retry in 1.361347502s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1
13482024-11-02T03:56:46.765ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655") failed: Property not found. retry in 1.282045624s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655
13492024-11-02T03:56:46.875ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723") failed: Property not found. retry in 1.308945593s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723
13502024-11-02T03:56:46.934ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6") failed: Property not found. retry in 749.56582ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6
13512024-11-02T03:56:46.962ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8") failed: Property not found. retry in 1.158130704s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8
13522024-11-02T03:56:46.983ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7f79c18f-7b53-4b8a-86fc-bb23b923ea1d") failed: Property not found. retry in 618.218991ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7f79c18f-7b53-4b8a-86fc-bb23b923ea1d
13532024-11-02T03:56:46.994ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338") failed: Property not found. retry in 1.37737574s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338
13542024-11-02T03:56:47.007ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_c2e0eec3-76f5-4824-aa40-15149bda9204") failed: Property not found. retry in 630.26338ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_c2e0eec3-76f5-4824-aa40-15149bda9204
13552024-11-02T03:56:47.128ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b") failed: Property not found. retry in 805.05849ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b
13562024-11-02T03:56:47.141ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_9325209d-81b8-49e4-be6a-cedff3ef78c1") failed: Property not found. retry in 1.462142224s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_9325209d-81b8-49e4-be6a-cedff3ef78c1
13572024-11-02T03:56:47.180ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16ad95ab-8ec8-4145-a12e-90482aa77a7f") failed: Property not found. retry in 1.386401799s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16ad95ab-8ec8-4145-a12e-90482aa77a7f
13582024-11-02T03:56:47.225ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_060b7a36-3ed1-4e47-89cf-3f46bfb09b6c") failed: Property not found. retry in 1.180485421s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_060b7a36-3ed1-4e47-89cf-3f46bfb09b6c
13592024-11-02T03:56:47.289ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d2bdfdcf-c904-463a-87e2-60b209fd743f") failed: Property not found. retry in 711.764827ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d2bdfdcf-c904-463a-87e2-60b209fd743f
13602024-11-02T03:56:47.346ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8bad491f-6df6-49cb-9ad0-7043316ba40c") failed: Property not found. retry in 508.01245ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8bad491f-6df6-49cb-9ad0-7043316ba40c
13612024-11-02T03:56:47.359ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8d16b3eb-bad3-42c3-aace-86cdfefc9878") failed: Property not found. retry in 1.332102416s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8d16b3eb-bad3-42c3-aace-86cdfefc9878
13622024-11-02T03:56:47.393ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0") failed: Property not found. retry in 890.468174ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0
13632024-11-02T03:56:47.615ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7f79c18f-7b53-4b8a-86fc-bb23b923ea1d") failed: Property not found. retry in 709.650841ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7f79c18f-7b53-4b8a-86fc-bb23b923ea1d
13642024-11-02T03:56:47.647ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f") failed: Property not found. retry in 1.223817076s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f
13652024-11-02T03:56:47.658ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6b885e4b-9eef-45af-9061-b78cac8479f0") failed: Property not found. retry in 1.274807134s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6b885e4b-9eef-45af-9061-b78cac8479f0
13662024-11-02T03:56:47.670ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_c2e0eec3-76f5-4824-aa40-15149bda9204") failed: Property not found. retry in 1.224746039s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_c2e0eec3-76f5-4824-aa40-15149bda9204
13672024-11-02T03:56:47.697ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6") failed: Property not found. retry in 1.206836059s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6
13682024-11-02T03:56:47.869ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8bad491f-6df6-49cb-9ad0-7043316ba40c") failed: Property not found. retry in 722.107814ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8bad491f-6df6-49cb-9ad0-7043316ba40c
13692024-11-02T03:56:47.947ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b") failed: Property not found. retry in 1.343858942s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b
13702024-11-02T03:56:48.014ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d2bdfdcf-c904-463a-87e2-60b209fd743f") failed: Property not found. retry in 812.227191ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d2bdfdcf-c904-463a-87e2-60b209fd743f
13712024-11-02T03:56:48.058ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655") failed: Property not found. retry in 1.289105838s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655
13722024-11-02T03:56:48.069ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1") failed: Property not found. retry in 1.286266683s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1
13732024-11-02T03:56:48.134ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8") failed: Property not found. retry in 863.625571ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8
13742024-11-02T03:56:48.196ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723") failed: Property not found. retry in 1.346642235s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723
13752024-11-02T03:56:48.296ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0") failed: Property not found. retry in 714.211209ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0
13762024-11-02T03:56:48.339ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7f79c18f-7b53-4b8a-86fc-bb23b923ea1d") failed: Property not found. retry in 534.11217ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7f79c18f-7b53-4b8a-86fc-bb23b923ea1d
13772024-11-02T03:56:48.389ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338") failed: Property not found. retry in 998.928061ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338
13782024-11-02T03:56:48.420ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_060b7a36-3ed1-4e47-89cf-3f46bfb09b6c") failed: Property not found. retry in 949.648315ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_060b7a36-3ed1-4e47-89cf-3f46bfb09b6c
13792024-11-02T03:56:48.587ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16ad95ab-8ec8-4145-a12e-90482aa77a7f") failed: Property not found. retry in 1.247593618s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16ad95ab-8ec8-4145-a12e-90482aa77a7f
13802024-11-02T03:56:48.606ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8bad491f-6df6-49cb-9ad0-7043316ba40c") failed: Property not found. retry in 1.290014468s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8bad491f-6df6-49cb-9ad0-7043316ba40c
13812024-11-02T03:56:48.618ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_9325209d-81b8-49e4-be6a-cedff3ef78c1") failed: Property not found. retry in 897.61364ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_9325209d-81b8-49e4-be6a-cedff3ef78c1
13822024-11-02T03:56:48.705ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8d16b3eb-bad3-42c3-aace-86cdfefc9878") failed: Property not found. retry in 1.409544341s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8d16b3eb-bad3-42c3-aace-86cdfefc9878
13832024-11-02T03:56:48.841ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d2bdfdcf-c904-463a-87e2-60b209fd743f") failed: Property not found. retry in 1.184634952s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d2bdfdcf-c904-463a-87e2-60b209fd743f
13842024-11-02T03:56:48.884ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f") failed: Property not found. retry in 838.97696ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f
13852024-11-02T03:56:48.895ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7f79c18f-7b53-4b8a-86fc-bb23b923ea1d") failed: Property not found. retry in 1.062364004s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7f79c18f-7b53-4b8a-86fc-bb23b923ea1d
13862024-11-02T03:56:48.909ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_c2e0eec3-76f5-4824-aa40-15149bda9204") failed: Property not found. retry in 1.2686274s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_c2e0eec3-76f5-4824-aa40-15149bda9204
13872024-11-02T03:56:48.920ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6") failed: Property not found. retry in 837.976613ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6
13882024-11-02T03:56:48.944ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6b885e4b-9eef-45af-9061-b78cac8479f0") failed: Property not found. retry in 944.565531ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6b885e4b-9eef-45af-9061-b78cac8479f0
13892024-11-02T03:56:49.010ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8") failed: Property not found. retry in 1.264110001s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8
13902024-11-02T03:56:49.020ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0") failed: Property not found. retry in 943.526181ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0
13912024-11-02T03:56:49.303ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b") failed: Property not found. retry in 1.430189985s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b
13922024-11-02T03:56:49.359ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655") failed: Property not found. retry in 569.20504ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655
13932024-11-02T03:56:49.368ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1") failed: Property not found. retry in 1.029275661s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1
13942024-11-02T03:56:49.378ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_060b7a36-3ed1-4e47-89cf-3f46bfb09b6c") failed: Property not found. retry in 1.334358134s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_060b7a36-3ed1-4e47-89cf-3f46bfb09b6c
13952024-11-02T03:56:49.398ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338") failed: Property not found. retry in 676.135751ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338
13962024-11-02T03:56:49.529ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_9325209d-81b8-49e4-be6a-cedff3ef78c1") failed: Property not found. retry in 1.228815585s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_9325209d-81b8-49e4-be6a-cedff3ef78c1
13972024-11-02T03:56:49.555ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723") failed: Property not found. retry in 501.463463ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723
13982024-11-02T03:56:49.736ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f") failed: Property not found. retry in 952.493317ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f
13992024-11-02T03:56:49.770ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6") failed: Property not found. retry in 933.749462ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6
14002024-11-02T03:56:49.846ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16ad95ab-8ec8-4145-a12e-90482aa77a7f") failed: Property not found. retry in 913.538344ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16ad95ab-8ec8-4145-a12e-90482aa77a7f
14012024-11-02T03:56:49.900ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6b885e4b-9eef-45af-9061-b78cac8479f0") failed: Property not found. retry in 621.800343ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6b885e4b-9eef-45af-9061-b78cac8479f0
14022024-11-02T03:56:49.912ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8bad491f-6df6-49cb-9ad0-7043316ba40c") failed: Property not found. retry in 836.77177ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8bad491f-6df6-49cb-9ad0-7043316ba40c
14032024-11-02T03:56:49.943ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655") failed: Property not found. retry in 1.31678311s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655
14042024-11-02T03:56:49.973ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7f79c18f-7b53-4b8a-86fc-bb23b923ea1d") failed: Property not found. retry in 697.144725ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7f79c18f-7b53-4b8a-86fc-bb23b923ea1d
14052024-11-02T03:56:49.988ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0") failed: Property not found. retry in 1.175195846s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0
14062024-11-02T03:56:50.036ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d2bdfdcf-c904-463a-87e2-60b209fd743f") failed: Property not found. retry in 935.350133ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d2bdfdcf-c904-463a-87e2-60b209fd743f
14072024-11-02T03:56:50.071ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723") failed: Property not found. retry in 948.53449ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723
14082024-11-02T03:56:50.087ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338") failed: Property not found. retry in 1.292594974s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338
14092024-11-02T03:56:50.124ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8d16b3eb-bad3-42c3-aace-86cdfefc9878") failed: Property not found. retry in 883.159581ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8d16b3eb-bad3-42c3-aace-86cdfefc9878
14102024-11-02T03:56:50.185ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_c2e0eec3-76f5-4824-aa40-15149bda9204") failed: Property not found. retry in 1.32539518s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_c2e0eec3-76f5-4824-aa40-15149bda9204
14112024-11-02T03:56:50.289ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8") failed: Property not found. retry in 1.035104142s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8
14122024-11-02T03:56:50.410ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1") failed: Property not found. retry in 859.232203ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1
14132024-11-02T03:56:50.533ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6b885e4b-9eef-45af-9061-b78cac8479f0") failed: Property not found. retry in 665.406265ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6b885e4b-9eef-45af-9061-b78cac8479f0
14142024-11-02T03:56:50.682ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7f79c18f-7b53-4b8a-86fc-bb23b923ea1d") failed: Property not found. retry in 1.215822144s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7f79c18f-7b53-4b8a-86fc-bb23b923ea1d
14152024-11-02T03:56:50.699ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f") failed: Property not found. retry in 570.437719ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f
14162024-11-02T03:56:50.717ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6") failed: Property not found. retry in 1.406781963s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6
14172024-11-02T03:56:50.729ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_060b7a36-3ed1-4e47-89cf-3f46bfb09b6c") failed: Property not found. retry in 941.078432ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_060b7a36-3ed1-4e47-89cf-3f46bfb09b6c
14182024-11-02T03:56:50.744ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b") failed: Property not found. retry in 905.638171ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b
14192024-11-02T03:56:50.762ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8bad491f-6df6-49cb-9ad0-7043316ba40c") failed: Property not found. retry in 1.227497459s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8bad491f-6df6-49cb-9ad0-7043316ba40c
14202024-11-02T03:56:50.772ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_9325209d-81b8-49e4-be6a-cedff3ef78c1") failed: Property not found. retry in 1.295204267s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_9325209d-81b8-49e4-be6a-cedff3ef78c1
14212024-11-02T03:56:50.782ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16ad95ab-8ec8-4145-a12e-90482aa77a7f") failed: Property not found. retry in 1.306912361s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16ad95ab-8ec8-4145-a12e-90482aa77a7f
14222024-11-02T03:56:50.985ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d2bdfdcf-c904-463a-87e2-60b209fd743f") failed: Property not found. retry in 1.318041509s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d2bdfdcf-c904-463a-87e2-60b209fd743f
14232024-11-02T03:56:51.022ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8d16b3eb-bad3-42c3-aace-86cdfefc9878") failed: Property not found. retry in 1.092216598s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8d16b3eb-bad3-42c3-aace-86cdfefc9878
14242024-11-02T03:56:51.035ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723") failed: Property not found. retry in 1.413105273s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723
14252024-11-02T03:56:51.178ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0") failed: Property not found. retry in 1.042166604s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0
14262024-11-02T03:56:51.212ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6b885e4b-9eef-45af-9061-b78cac8479f0") failed: Property not found. retry in 981.837726ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6b885e4b-9eef-45af-9061-b78cac8479f0
14272024-11-02T03:56:51.276ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655") failed: Property not found. retry in 1.162784415s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655
14282024-11-02T03:56:51.289ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f") failed: Property not found. retry in 763.787232ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f
14292024-11-02T03:56:51.301ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1") failed: Property not found. retry in 610.823705ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1
14302024-11-02T03:56:51.340ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8") failed: Property not found. retry in 521.219738ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8
14312024-11-02T03:56:51.399ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338") failed: Property not found. retry in 823.89586ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338
14322024-11-02T03:56:51.510ZINFOSledAgent (ServiceManager): Profile for oxz_clickhouse_f2d1f946-a521-4b2f-ab66-945d3b5d0ac5: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService20'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::e'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/clickhouse"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::e'/> <propval type="astring" name="listen_port" value='8123'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/clickhouse-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::e]:8888'/> <propval type="astring" name="ch_address" value='[fd00:1122:3344:101::e]:8123'/> <propval type="astring" name="ch_binary" value='/opt/oxide/clickhouse/clickhouse'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14332024-11-02T03:56:51.511ZINFOSledAgent (ServiceManager): Booting oxz_clickhouse_f2d1f946-a521-4b2f-ab66-945d3b5d0ac5 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_clickhouse_f2d1f946-a521-4b2f-ab66-945d3b5d0ac5
14342024-11-02T03:56:51.534ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_c2e0eec3-76f5-4824-aa40-15149bda9204") failed: Property not found. retry in 960.017425ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_c2e0eec3-76f5-4824-aa40-15149bda9204
14352024-11-02T03:56:51.666ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b") failed: Property not found. retry in 1.457490232s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b
14362024-11-02T03:56:51.688ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_060b7a36-3ed1-4e47-89cf-3f46bfb09b6c") failed: Property not found. retry in 1.131450878s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_060b7a36-3ed1-4e47-89cf-3f46bfb09b6c
14372024-11-02T03:56:51.768ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_f2d1f946-a521-4b2f-ab66-945d3b5d0ac5") failed: Property not found. retry in 51.91276ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_f2d1f946-a521-4b2f-ab66-945d3b5d0ac5
14382024-11-02T03:56:51.860ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_f2d1f946-a521-4b2f-ab66-945d3b5d0ac5") failed: Property not found. retry in 123.501736ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_f2d1f946-a521-4b2f-ab66-945d3b5d0ac5
14392024-11-02T03:56:51.883ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8") failed: Property not found. retry in 1.423626939s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8
14402024-11-02T03:56:52.066ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7f79c18f-7b53-4b8a-86fc-bb23b923ea1d") failed: Property not found. retry in 1.499149782s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7f79c18f-7b53-4b8a-86fc-bb23b923ea1d
14412024-11-02T03:56:52.128ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8bad491f-6df6-49cb-9ad0-7043316ba40c") failed: Property not found. retry in 986.654666ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8bad491f-6df6-49cb-9ad0-7043316ba40c
14422024-11-02T03:56:52.143ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6") failed: Property not found. retry in 685.89985ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6
14432024-11-02T03:56:52.154ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_9325209d-81b8-49e4-be6a-cedff3ef78c1") failed: Property not found. retry in 772.38932ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_9325209d-81b8-49e4-be6a-cedff3ef78c1
14442024-11-02T03:56:52.163ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f") failed: Property not found. retry in 1.08760271s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f
14452024-11-02T03:56:52.174ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16ad95ab-8ec8-4145-a12e-90482aa77a7f") failed: Property not found. retry in 1.214506099s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16ad95ab-8ec8-4145-a12e-90482aa77a7f
14462024-11-02T03:56:52.187ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_f2d1f946-a521-4b2f-ab66-945d3b5d0ac5") failed: Property not found. retry in 298.31183ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_f2d1f946-a521-4b2f-ab66-945d3b5d0ac5
14472024-11-02T03:56:52.199ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1") failed: Property not found. retry in 1.395149099s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1
14482024-11-02T03:56:52.209ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8d16b3eb-bad3-42c3-aace-86cdfefc9878") failed: Property not found. retry in 592.162231ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8d16b3eb-bad3-42c3-aace-86cdfefc9878
14492024-11-02T03:56:52.220ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6b885e4b-9eef-45af-9061-b78cac8479f0") failed: Property not found. retry in 1.290809725s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6b885e4b-9eef-45af-9061-b78cac8479f0
14502024-11-02T03:56:52.231ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0") failed: Property not found. retry in 1.129558789s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0
14512024-11-02T03:56:52.239ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338") failed: Property not found. retry in 607.59069ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338
14522024-11-02T03:56:52.313ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d2bdfdcf-c904-463a-87e2-60b209fd743f") failed: Property not found. retry in 1.483099952s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d2bdfdcf-c904-463a-87e2-60b209fd743f
14532024-11-02T03:56:52.449ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655") failed: Property not found. retry in 1.435825214s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655
14542024-11-02T03:56:52.459ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723") failed: Property not found. retry in 1.087300262s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723
14552024-11-02T03:56:52.495ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_f2d1f946-a521-4b2f-ab66-945d3b5d0ac5") failed: Property not found. retry in 325.342317ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_f2d1f946-a521-4b2f-ab66-945d3b5d0ac5
14562024-11-02T03:56:52.506ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_c2e0eec3-76f5-4824-aa40-15149bda9204") failed: Property not found. retry in 837.189484ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_c2e0eec3-76f5-4824-aa40-15149bda9204
14572024-11-02T03:56:52.812ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8d16b3eb-bad3-42c3-aace-86cdfefc9878") failed: Property not found. retry in 1.375145457s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8d16b3eb-bad3-42c3-aace-86cdfefc9878
14582024-11-02T03:56:52.830ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_060b7a36-3ed1-4e47-89cf-3f46bfb09b6c") failed: Property not found. retry in 1.284235835s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_060b7a36-3ed1-4e47-89cf-3f46bfb09b6c
14592024-11-02T03:56:52.839ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6") failed: Property not found. retry in 1.257434065s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6
14602024-11-02T03:56:52.850ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_f2d1f946-a521-4b2f-ab66-945d3b5d0ac5") failed: Property not found. retry in 490.916857ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_f2d1f946-a521-4b2f-ab66-945d3b5d0ac5
14612024-11-02T03:56:52.860ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338") failed: Property not found. retry in 977.892796ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338
14622024-11-02T03:56:52.936ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_9325209d-81b8-49e4-be6a-cedff3ef78c1") failed: Property not found. retry in 1.493737799s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_9325209d-81b8-49e4-be6a-cedff3ef78c1
14632024-11-02T03:56:53.125ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8bad491f-6df6-49cb-9ad0-7043316ba40c") failed: Property not found. retry in 882.718764ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8bad491f-6df6-49cb-9ad0-7043316ba40c
14642024-11-02T03:56:53.135ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b") failed: Property not found. retry in 1.296863178s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b
14652024-11-02T03:56:53.262ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f") failed: Property not found. retry in 754.4761ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f
14662024-11-02T03:56:53.317ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8") failed: Property not found. retry in 1.119249507s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8
14672024-11-02T03:56:53.351ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_f2d1f946-a521-4b2f-ab66-945d3b5d0ac5") failed: Property not found. retry in 1.245262984s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_f2d1f946-a521-4b2f-ab66-945d3b5d0ac5
14682024-11-02T03:56:53.360ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_c2e0eec3-76f5-4824-aa40-15149bda9204") failed: Property not found. retry in 732.119811ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_c2e0eec3-76f5-4824-aa40-15149bda9204
14692024-11-02T03:56:53.371ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0") failed: Property not found. retry in 805.99562ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0
14702024-11-02T03:56:53.399ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16ad95ab-8ec8-4145-a12e-90482aa77a7f") failed: Property not found. retry in 1.293357268s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16ad95ab-8ec8-4145-a12e-90482aa77a7f
14712024-11-02T03:56:53.522ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6b885e4b-9eef-45af-9061-b78cac8479f0") failed: Property not found. retry in 1.019173815s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6b885e4b-9eef-45af-9061-b78cac8479f0
14722024-11-02T03:56:53.557ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723") failed: Property not found. retry in 808.560876ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723
14732024-11-02T03:56:53.575ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7f79c18f-7b53-4b8a-86fc-bb23b923ea1d") failed: Property not found. retry in 929.043216ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7f79c18f-7b53-4b8a-86fc-bb23b923ea1d
14742024-11-02T03:56:53.605ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1") failed: Property not found. retry in 1.12484627s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1
14752024-11-02T03:56:53.810ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d2bdfdcf-c904-463a-87e2-60b209fd743f") failed: Property not found. retry in 1.320312383s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d2bdfdcf-c904-463a-87e2-60b209fd743f
14762024-11-02T03:56:53.852ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338") failed: Property not found. retry in 975.930978ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338
14772024-11-02T03:56:53.898ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655") failed: Property not found. retry in 873.675824ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655
14782024-11-02T03:56:54.022ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8bad491f-6df6-49cb-9ad0-7043316ba40c") failed: Property not found. retry in 1.374231266s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8bad491f-6df6-49cb-9ad0-7043316ba40c
14792024-11-02T03:56:54.035ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f") failed: Property not found. retry in 1.322240139s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f
14802024-11-02T03:56:54.106ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_c2e0eec3-76f5-4824-aa40-15149bda9204") failed: Property not found. retry in 1.309421317s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_c2e0eec3-76f5-4824-aa40-15149bda9204
14812024-11-02T03:56:54.124ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6") failed: Property not found. retry in 1.471882441s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6
14822024-11-02T03:56:54.140ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_060b7a36-3ed1-4e47-89cf-3f46bfb09b6c") failed: Property not found. retry in 579.410067ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_060b7a36-3ed1-4e47-89cf-3f46bfb09b6c
14832024-11-02T03:56:54.192ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0") failed: Property not found. retry in 612.085921ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0
14842024-11-02T03:56:54.208ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8d16b3eb-bad3-42c3-aace-86cdfefc9878") failed: Property not found. retry in 1.217870064s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8d16b3eb-bad3-42c3-aace-86cdfefc9878
14852024-11-02T03:56:54.383ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723") failed: Property not found. retry in 1.09792635s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723
14862024-11-02T03:56:54.457ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_9325209d-81b8-49e4-be6a-cedff3ef78c1") failed: Property not found. retry in 1.167780732s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_9325209d-81b8-49e4-be6a-cedff3ef78c1
14872024-11-02T03:56:54.478ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8") failed: Property not found. retry in 1.468366117s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8
14882024-11-02T03:56:54.497ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b") failed: Property not found. retry in 1.456355744s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b
14892024-11-02T03:56:54.535ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7f79c18f-7b53-4b8a-86fc-bb23b923ea1d") failed: Property not found. retry in 1.280271558s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7f79c18f-7b53-4b8a-86fc-bb23b923ea1d
14902024-11-02T03:56:54.569ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6b885e4b-9eef-45af-9061-b78cac8479f0") failed: Property not found. retry in 1.492390646s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6b885e4b-9eef-45af-9061-b78cac8479f0
14912024-11-02T03:56:54.626ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_f2d1f946-a521-4b2f-ab66-945d3b5d0ac5") failed: Property not found. retry in 668.577889ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_f2d1f946-a521-4b2f-ab66-945d3b5d0ac5
14922024-11-02T03:56:54.720ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16ad95ab-8ec8-4145-a12e-90482aa77a7f") failed: Property not found. retry in 1.460807348s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16ad95ab-8ec8-4145-a12e-90482aa77a7f
14932024-11-02T03:56:54.743ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_060b7a36-3ed1-4e47-89cf-3f46bfb09b6c") failed: Property not found. retry in 1.181077719s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_060b7a36-3ed1-4e47-89cf-3f46bfb09b6c
14942024-11-02T03:56:54.765ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1") failed: Property not found. retry in 646.131457ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1
14952024-11-02T03:56:54.790ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655") failed: Property not found. retry in 1.287269701s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655
14962024-11-02T03:56:54.821ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0") failed: Property not found. retry in 1.240572771s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0
14972024-11-02T03:56:54.845ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338") failed: Property not found. retry in 815.511354ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338
14982024-11-02T03:56:55.145ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d2bdfdcf-c904-463a-87e2-60b209fd743f") failed: Property not found. retry in 803.738336ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d2bdfdcf-c904-463a-87e2-60b209fd743f
14992024-11-02T03:56:55.310ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_f2d1f946-a521-4b2f-ab66-945d3b5d0ac5") failed: Property not found. retry in 1.462870179s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_f2d1f946-a521-4b2f-ab66-945d3b5d0ac5
15002024-11-02T03:56:55.370ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f") failed: Property not found. retry in 917.552507ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f
15012024-11-02T03:56:55.412ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8bad491f-6df6-49cb-9ad0-7043316ba40c") failed: Property not found. retry in 1.479396496s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8bad491f-6df6-49cb-9ad0-7043316ba40c
15022024-11-02T03:56:55.422ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1") failed: Property not found. retry in 1.43427194s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1
15032024-11-02T03:56:55.434ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_c2e0eec3-76f5-4824-aa40-15149bda9204") failed: Property not found. retry in 1.116840298s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_c2e0eec3-76f5-4824-aa40-15149bda9204
15042024-11-02T03:56:55.449ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8d16b3eb-bad3-42c3-aace-86cdfefc9878") failed: Property not found. retry in 1.350520988s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8d16b3eb-bad3-42c3-aace-86cdfefc9878
15052024-11-02T03:56:55.497ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723") failed: Property not found. retry in 648.328936ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723
15062024-11-02T03:56:55.609ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6") failed: Property not found. retry in 553.69928ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6
15072024-11-02T03:56:55.639ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_9325209d-81b8-49e4-be6a-cedff3ef78c1") failed: Property not found. retry in 980.26596ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_9325209d-81b8-49e4-be6a-cedff3ef78c1
15082024-11-02T03:56:55.672ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338") failed: Property not found. retry in 1.44387266s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338
15092024-11-02T03:56:55.833ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7f79c18f-7b53-4b8a-86fc-bb23b923ea1d") failed: Property not found. retry in 1.103493639s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7f79c18f-7b53-4b8a-86fc-bb23b923ea1d
15102024-11-02T03:56:55.939ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_060b7a36-3ed1-4e47-89cf-3f46bfb09b6c") failed: Property not found. retry in 949.426049ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_060b7a36-3ed1-4e47-89cf-3f46bfb09b6c
15112024-11-02T03:56:55.962ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8") failed: Property not found. retry in 701.930882ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8
15122024-11-02T03:56:55.974ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b") failed: Property not found. retry in 1.497695795s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b
15132024-11-02T03:56:55.990ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d2bdfdcf-c904-463a-87e2-60b209fd743f") failed: Property not found. retry in 880.697989ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d2bdfdcf-c904-463a-87e2-60b209fd743f
15142024-11-02T03:56:56.076ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6b885e4b-9eef-45af-9061-b78cac8479f0") failed: Property not found. retry in 1.407024313s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6b885e4b-9eef-45af-9061-b78cac8479f0
15152024-11-02T03:56:56.088ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0") failed: Property not found. retry in 1.217850434s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0
15162024-11-02T03:56:56.103ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655") failed: Property not found. retry in 630.609591ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655
15172024-11-02T03:56:56.163ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723") failed: Property not found. retry in 607.551441ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723
15182024-11-02T03:56:56.176ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6") failed: Property not found. retry in 626.813149ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6
15192024-11-02T03:56:56.198ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16ad95ab-8ec8-4145-a12e-90482aa77a7f") failed: Property not found. retry in 621.442681ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16ad95ab-8ec8-4145-a12e-90482aa77a7f
15202024-11-02T03:56:56.303ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f") failed: Property not found. retry in 1.147782944s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f
15212024-11-02T03:56:56.561ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_c2e0eec3-76f5-4824-aa40-15149bda9204") failed: Property not found. retry in 1.336496408s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_c2e0eec3-76f5-4824-aa40-15149bda9204
15222024-11-02T03:56:56.629ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_9325209d-81b8-49e4-be6a-cedff3ef78c1") failed: Property not found. retry in 762.699877ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_9325209d-81b8-49e4-be6a-cedff3ef78c1
15232024-11-02T03:56:56.674ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8") failed: Property not found. retry in 746.760868ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8
15242024-11-02T03:56:56.744ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655") failed: Property not found. retry in 1.129618002s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655
15252024-11-02T03:56:56.780ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723") failed: Property not found. retry in 674.043524ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723
15262024-11-02T03:56:56.798ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_f2d1f946-a521-4b2f-ab66-945d3b5d0ac5") failed: Property not found. retry in 639.7288ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_f2d1f946-a521-4b2f-ab66-945d3b5d0ac5
15272024-11-02T03:56:56.812ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8d16b3eb-bad3-42c3-aace-86cdfefc9878") failed: Property not found. retry in 735.960293ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8d16b3eb-bad3-42c3-aace-86cdfefc9878
15282024-11-02T03:56:56.823ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6") failed: Property not found. retry in 780.188095ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6
15292024-11-02T03:56:56.832ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16ad95ab-8ec8-4145-a12e-90482aa77a7f") failed: Property not found. retry in 1.296159021s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16ad95ab-8ec8-4145-a12e-90482aa77a7f
15302024-11-02T03:56:56.865ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1") failed: Property not found. retry in 959.021611ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1
15312024-11-02T03:56:56.880ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d2bdfdcf-c904-463a-87e2-60b209fd743f") failed: Property not found. retry in 712.815867ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d2bdfdcf-c904-463a-87e2-60b209fd743f
15322024-11-02T03:56:56.896ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_060b7a36-3ed1-4e47-89cf-3f46bfb09b6c") failed: Property not found. retry in 670.553585ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_060b7a36-3ed1-4e47-89cf-3f46bfb09b6c
15332024-11-02T03:56:56.904ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8bad491f-6df6-49cb-9ad0-7043316ba40c") failed: Property not found. retry in 1.441039589s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8bad491f-6df6-49cb-9ad0-7043316ba40c
15342024-11-02T03:56:56.949ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7f79c18f-7b53-4b8a-86fc-bb23b923ea1d") failed: Property not found. retry in 789.538558ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7f79c18f-7b53-4b8a-86fc-bb23b923ea1d
15352024-11-02T03:56:57.127ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338") failed: Property not found. retry in 1.089797533s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338
15362024-11-02T03:56:57.318ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0") failed: Property not found. retry in 674.398476ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0
15372024-11-02T03:56:57.402ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_9325209d-81b8-49e4-be6a-cedff3ef78c1") failed: Property not found. retry in 816.860024ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_9325209d-81b8-49e4-be6a-cedff3ef78c1
15382024-11-02T03:56:57.431ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8") failed: Property not found. retry in 1.283567993s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8
15392024-11-02T03:56:57.448ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_f2d1f946-a521-4b2f-ab66-945d3b5d0ac5") failed: Property not found. retry in 710.190359ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_f2d1f946-a521-4b2f-ab66-945d3b5d0ac5
15402024-11-02T03:56:57.460ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f") failed: Property not found. retry in 769.53307ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f
15412024-11-02T03:56:57.469ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723") failed: Property not found. retry in 570.341219ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723
15422024-11-02T03:56:57.482ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b") failed: Property not found. retry in 936.203113ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b
15432024-11-02T03:56:57.493ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6b885e4b-9eef-45af-9061-b78cac8479f0") failed: Property not found. retry in 1.004899372s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6b885e4b-9eef-45af-9061-b78cac8479f0
15442024-11-02T03:56:57.558ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8d16b3eb-bad3-42c3-aace-86cdfefc9878") failed: Property not found. retry in 1.327988253s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8d16b3eb-bad3-42c3-aace-86cdfefc9878
15452024-11-02T03:56:57.576ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_060b7a36-3ed1-4e47-89cf-3f46bfb09b6c") failed: Property not found. retry in 680.154995ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_060b7a36-3ed1-4e47-89cf-3f46bfb09b6c
15462024-11-02T03:56:57.601ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_d2bdfdcf-c904-463a-87e2-60b209fd743f") failed: Property not found. retry in 937.443427ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_d2bdfdcf-c904-463a-87e2-60b209fd743f
15472024-11-02T03:56:57.613ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6") failed: Property not found. retry in 610.125255ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6
15482024-11-02T03:56:57.752ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7f79c18f-7b53-4b8a-86fc-bb23b923ea1d") failed: Property not found. retry in 1.497750905s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7f79c18f-7b53-4b8a-86fc-bb23b923ea1d
15492024-11-02T03:56:57.836ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1") failed: Property not found. retry in 1.476489914s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1
15502024-11-02T03:56:57.885ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655") failed: Property not found. retry in 863.602298ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655
15512024-11-02T03:56:57.909ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_c2e0eec3-76f5-4824-aa40-15149bda9204") failed: Property not found. retry in 828.235614ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_c2e0eec3-76f5-4824-aa40-15149bda9204
15522024-11-02T03:56:58.054ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723") failed: Property not found. retry in 1.088720847s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723
15532024-11-02T03:56:58.139ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_16ad95ab-8ec8-4145-a12e-90482aa77a7f") failed: Property not found. retry in 1.220736593s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_16ad95ab-8ec8-4145-a12e-90482aa77a7f
15542024-11-02T03:56:58.168ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_f2d1f946-a521-4b2f-ab66-945d3b5d0ac5") failed: Property not found. retry in 986.668705ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_f2d1f946-a521-4b2f-ab66-945d3b5d0ac5
15552024-11-02T03:56:58.226ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338") failed: Property not found. retry in 1.192329161s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338
15562024-11-02T03:56:58.236ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6") failed: Property not found. retry in 823.722227ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6
15572024-11-02T03:56:58.245ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_9325209d-81b8-49e4-be6a-cedff3ef78c1") failed: Property not found. retry in 561.527669ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_9325209d-81b8-49e4-be6a-cedff3ef78c1
15582024-11-02T03:56:58.257ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f") failed: Property not found. retry in 1.028895532s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f
15592024-11-02T03:56:58.431ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b") failed: Property not found. retry in 531.387545ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b
15602024-11-02T03:56:59.233ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_f2d1f946-a521-4b2f-ab66-945d3b5d0ac5") failed: Property not found. retry in 1.218344857s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_f2d1f946-a521-4b2f-ab66-945d3b5d0ac5
15612024-11-02T03:56:59.623ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:35115
15622024-11-02T03:56:59.630ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2702 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:35115 req_id = fb5b6a44-6c97-43ba-9f5b-ad0724c21853 response_code = 200 uri = /baseboard
15632024-11-02T03:57:00.478ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_f2d1f946-a521-4b2f-ab66-945d3b5d0ac5") failed: Property not found. retry in 793.341322ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_f2d1f946-a521-4b2f-ab66-945d3b5d0ac5
15642024-11-02T03:57:01.321ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3273 zone = oxz_crucible_b214f328-1474-4895-bf12-1b9492fb7338
15652024-11-02T03:57:01.321ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3273 zone = oxz_crucible_8d16b3eb-bad3-42c3-aace-86cdfefc9878
15662024-11-02T03:57:01.321ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3273 zone = oxz_crucible_pantry_d792d4a3-6908-4d88-bf95-5371dd31e9d8
15672024-11-02T03:57:01.321ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3273 zone = oxz_crucible_6b885e4b-9eef-45af-9061-b78cac8479f0
15682024-11-02T03:57:01.321ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3273 zone = oxz_crucible_8bad491f-6df6-49cb-9ad0-7043316ba40c
15692024-11-02T03:57:01.321ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3273 zone = oxz_crucible_33738c1e-05b4-47e5-9843-07c78c3a10c6
15702024-11-02T03:57:01.321ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3273 zone = oxz_external_dns_f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b
15712024-11-02T03:57:01.321ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3273 zone = oxz_external_dns_7f79c18f-7b53-4b8a-86fc-bb23b923ea1d
15722024-11-02T03:57:01.321ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3273 zone = oxz_nexus_9325209d-81b8-49e4-be6a-cedff3ef78c1
15732024-11-02T03:57:01.321ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3273 zone = oxz_crucible_9a8e4d7b-b87b-44df-95d9-4efe041c611f
15742024-11-02T03:57:01.321ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3273 zone = oxz_crucible_16ad95ab-8ec8-4145-a12e-90482aa77a7f
15752024-11-02T03:57:01.321ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3273 zone = oxz_oximeter_3975424e-54cf-46ad-bdc1-6b8ad0a2e4f0
15762024-11-02T03:57:01.321ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3273 zone = oxz_crucible_ef8a820a-84e9-4141-b6bc-002a3603a655
15772024-11-02T03:57:01.321ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3273 zone = oxz_nexus_d2bdfdcf-c904-463a-87e2-60b209fd743f
15782024-11-02T03:57:01.321ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3273 zone = oxz_clickhouse_f2d1f946-a521-4b2f-ab66-945d3b5d0ac5
15792024-11-02T03:57:01.321ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3273 zone = oxz_crucible_325eee21-9fb8-4009-ab47-7f3e7e0fa723
15802024-11-02T03:57:01.321ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3273 zone = oxz_crucible_pantry_0ac842d5-c5fe-4985-b91a-beeb82623eb1
15812024-11-02T03:57:01.321ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3273 zone = oxz_crucible_pantry_c2e0eec3-76f5-4824-aa40-15149bda9204
15822024-11-02T03:57:01.321ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3273 zone = oxz_nexus_060b7a36-3ed1-4e47-89cf-3f46bfb09b6c
15832024-11-02T03:57:01.324ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/6308f1ec-c08e-4282-a1c2-db37d6b8a044/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
15842024-11-02T03:57:01.329ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/fda73319-9c6d-44d8-9377-b69537374c59/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
15852024-11-02T03:57:01.337ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 32927506 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:49646 req_id = 1db9c3b6-0a0a-402e-93f6-8d041cab5d38 response_code = 204 uri = /omicron-zones
15862024-11-02T03:57:01.347ZINFOSledAgent (RSS): Finished setting up services file = sled-agent/src/rack_setup/service.rs:1331
15872024-11-02T03:57:01.348ZINFOSledAgent (RSS): Writing ledger to /pool/int/6308f1ec-c08e-4282-a1c2-db37d6b8a044/config/.rss-plan-completed.marker.tmp file = common/src/ledger.rs:199
15882024-11-02T03:57:01.348ZINFOSledAgent (RSS): Writing ledger to /pool/int/fda73319-9c6d-44d8-9377-b69537374c59/config/.rss-plan-completed.marker.tmp file = common/src/ledger.rs:199
15892024-11-02T03:57:01.359ZINFOSledAgent (RSS): Handing off control to Nexus file = sled-agent/src/rack_setup/service.rs:708
15902024-11-02T03:57:01.367ZINFOSledAgent (RSS): Nexus address: [fd00:1122:3344:101::c]:12221 file = sled-agent/src/rack_setup/service.rs:720
15912024-11-02T03:57:01.430ZINFOSledAgent (RSS): rack_network_config: RackNetworkConfigV2 { bfd: [], bgp: [], infra_ip_first: 10.151.1.174, infra_ip_last: 10.151.1.174, ports: [ PortConfigV2 { addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.1.174, width: 24, }, ), vlan_id: None, }, ], autoneg: false, bgp_peers: [], lldp: None, port: "qsfp0", routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), nexthop: 10.151.1.100, rib_priority: None, vlan_id: None, }, ], switch: Switch0, tx_eq: None, uplink_port_fec: None, uplink_port_speed: Speed40G, }, ], rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, } file = sled-agent/src/rack_setup/service.rs:896
15922024-11-02T03:57:01.437ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/e190f004-2376-4aae-8296-5a5757dc1123/initialization-complete) file = sled-agent/src/rack_setup/service.rs:958
15932024-11-02T03:57:01.538ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/e190f004-2376-4aae-8296-5a5757dc1123/initialization-complete) file = sled-agent/src/rack_setup/service.rs:958
15942024-11-02T03:57:01.627ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/e190f004-2376-4aae-8296-5a5757dc1123/initialization-complete) file = sled-agent/src/rack_setup/service.rs:958
15952024-11-02T03:57:01.830ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/e190f004-2376-4aae-8296-5a5757dc1123/initialization-complete) file = sled-agent/src/rack_setup/service.rs:958
15962024-11-02T03:57:02.005ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/e190f004-2376-4aae-8296-5a5757dc1123/initialization-complete) file = sled-agent/src/rack_setup/service.rs:958
15972024-11-02T03:57:02.159ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 4.639258875s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:424 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
15982024-11-02T03:57:02.162ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/e190f004-2376-4aae-8296-5a5757dc1123/initialization-complete) file = sled-agent/src/rack_setup/service.rs:958
15992024-11-02T03:57:02.394ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/e190f004-2376-4aae-8296-5a5757dc1123/initialization-complete) file = sled-agent/src/rack_setup/service.rs:958
16002024-11-02T03:57:02.653ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/e190f004-2376-4aae-8296-5a5757dc1123/initialization-complete) file = sled-agent/src/rack_setup/service.rs:958
16012024-11-02T03:57:02.861ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/e190f004-2376-4aae-8296-5a5757dc1123/initialization-complete) file = sled-agent/src/rack_setup/service.rs:958
16022024-11-02T03:57:03.375ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/e190f004-2376-4aae-8296-5a5757dc1123/initialization-complete) file = sled-agent/src/rack_setup/service.rs:958
16032024-11-02T03:57:04.103ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/e190f004-2376-4aae-8296-5a5757dc1123/initialization-complete) file = sled-agent/src/rack_setup/service.rs:958
16042024-11-02T03:57:04.548ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/e190f004-2376-4aae-8296-5a5757dc1123/initialization-complete) file = sled-agent/src/rack_setup/service.rs:958
16052024-11-02T03:57:05.507ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/e190f004-2376-4aae-8296-5a5757dc1123/initialization-complete) file = sled-agent/src/rack_setup/service.rs:958
16062024-11-02T03:57:06.600ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/e190f004-2376-4aae-8296-5a5757dc1123/initialization-complete) file = sled-agent/src/rack_setup/service.rs:958
16072024-11-02T03:57:06.845ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 4.721337633s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:424 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
16082024-11-02T03:57:07.447ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/e190f004-2376-4aae-8296-5a5757dc1123/initialization-complete) file = sled-agent/src/rack_setup/service.rs:958
16092024-11-02T03:57:08.505ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/e190f004-2376-4aae-8296-5a5757dc1123/initialization-complete) file = sled-agent/src/rack_setup/service.rs:958
16102024-11-02T03:57:09.373ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/e190f004-2376-4aae-8296-5a5757dc1123/initialization-complete) file = sled-agent/src/rack_setup/service.rs:958
16112024-11-02T03:57:10.720ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/e190f004-2376-4aae-8296-5a5757dc1123/initialization-complete) file = sled-agent/src/rack_setup/service.rs:958
16122024-11-02T03:57:11.627ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 17.417287927s error = "Error Response: status: 503 Service Unavailable; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"a7e80e0e-0d2a-4255-9ce6-bb00b94f5ec1\\", \\"content-length\\": \\"133\\", \\"date\\": \\"Sat, 02 Nov 2024 03:57:10 GMT\\"}; value: Error { error_code: Some(\\"ServiceNotAvailable\\"), message: \\"Service Unavailable\\", request_id: \\"a7e80e0e-0d2a-4255-9ce6-bb00b94f5ec1\\" }" file = oximeter/producer/src/lib.rs:424 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
16132024-11-02T03:57:14.510ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "cfdeb94f-38c5-457b-8fb3-aee0fc6a0224", "content-length": "173", "date": "Sat, 02 Nov 2024 03:57:13 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"5b8cdd5a-f021-45f2-8323-b6c8f9776060\"", request_id: "cfdeb94f-38c5-457b-8fb3-aee0fc6a0224" } file = sled-agent/src/rack_setup/service.rs:958
16142024-11-02T03:57:17.336ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "01ca592f-5440-445d-a979-8a5cbbdfbb32", "content-length": "173", "date": "Sat, 02 Nov 2024 03:57:17 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"5b8cdd5a-f021-45f2-8323-b6c8f9776060\"", request_id: "01ca592f-5440-445d-a979-8a5cbbdfbb32" } file = sled-agent/src/rack_setup/service.rs:958
16152024-11-02T03:57:21.933ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "f64f98e3-2ea1-4280-b4da-58388ac878a1", "content-length": "173", "date": "Sat, 02 Nov 2024 03:57:21 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"5b8cdd5a-f021-45f2-8323-b6c8f9776060\"", request_id: "f64f98e3-2ea1-4280-b4da-58388ac878a1" } file = sled-agent/src/rack_setup/service.rs:958
16162024-11-02T03:57:26.891ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "96a1d58c-d623-4a75-a383-618007065dcb", "content-length": "173", "date": "Sat, 02 Nov 2024 03:57:26 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"5b8cdd5a-f021-45f2-8323-b6c8f9776060\"", request_id: "96a1d58c-d623-4a75-a383-618007065dcb" } file = sled-agent/src/rack_setup/service.rs:958
16172024-11-02T03:57:29.664ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:59846
16182024-11-02T03:57:29.664ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 31 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:59846 req_id = eb62f1d0-8787-444e-80e1-a47138d2c7a0 response_code = 200 uri = /baseboard
16192024-11-02T03:57:31.365ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "1ad493fa-cb3a-44a4-84f8-6944ae21fea4", "content-length": "173", "date": "Sat, 02 Nov 2024 03:57:31 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"5b8cdd5a-f021-45f2-8323-b6c8f9776060\"", request_id: "1ad493fa-cb3a-44a4-84f8-6944ae21fea4" } file = sled-agent/src/rack_setup/service.rs:958
16202024-11-02T03:57:37.945ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "93ac91ad-7fd5-401c-bbd2-1598bd088efa", "content-length": "173", "date": "Sat, 02 Nov 2024 03:57:37 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"5b8cdd5a-f021-45f2-8323-b6c8f9776060\"", request_id: "93ac91ad-7fd5-401c-bbd2-1598bd088efa" } file = sled-agent/src/rack_setup/service.rs:958
16212024-11-02T03:57:43.827ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "d9a65a1c-2c51-478c-9478-589bda819d34", "content-length": "173", "date": "Sat, 02 Nov 2024 03:57:43 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"5b8cdd5a-f021-45f2-8323-b6c8f9776060\"", request_id: "d9a65a1c-2c51-478c-9478-589bda819d34" } file = sled-agent/src/rack_setup/service.rs:958
16222024-11-02T03:57:43.946ZINFOSledAgent (dropshot): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:56897 remote_addr = [fd00:1122:3344:101::d]:65252 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
16232024-11-02T03:57:43.972ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 23369 local_addr = [fd00:1122:3344:101::1]:56897 method = GET remote_addr = [fd00:1122:3344:101::d]:65252 req_id = f57c343e-61ce-40da-8efb-dd94a42cf792 response_code = 200 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 uri = /5b8cdd5a-f021-45f2-8323-b6c8f9776060
16242024-11-02T03:57:50.275ZINFOSledAgent (NexusNotifierTask): Successfully put SledAgentInfo to nexus file = sled-agent/src/nexus.rs:500 new_generation = 1 old_generation = 1 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
16252024-11-02T03:57:50.363ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:39487
16262024-11-02T03:57:50.366ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
16272024-11-02T03:57:50.366ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
16282024-11-02T03:57:50.367ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/6308f1ec-c08e-4282-a1c2-db37d6b8a044/config/omicron-zones.json file = common/src/ledger.rs:177
16292024-11-02T03:57:50.369ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/fda73319-9c6d-44d8-9377-b69537374c59/config/omicron-zones.json file = common/src/ledger.rs:177
16302024-11-02T03:57:50.557ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 193121 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:39487 req_id = fcd318c1-b9f9-4a19-b289-8b46c5fe9033 response_code = 200 uri = /inventory
16312024-11-02T03:57:55.855ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:48591
16322024-11-02T03:57:55.859ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:731 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 27d7c9ec-b8a8-4b31-ae12-c49ad36788f1, kind: Service { id: 7f79c18f-7b53-4b8a-86fc-bb23b923ea1d }, name: Name("external-dns-7f79c18f-7b53-4b8a-86fc-bb23b923ea1d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 238, 20])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 29b92946-7588-4b89-8147-322049e1f231, kind: Service { id: f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b }, name: Name("external-dns-f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 239, 77])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0cd6826d-84a0-4cd5-b7c6-32cb83627e6c, kind: Service { id: 9325209d-81b8-49e4-be6a-cedff3ef78c1 }, name: Name("nexus-9325209d-81b8-49e4-be6a-cedff3ef78c1"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 192, 104])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ec778b7a-3947-4776-98f2-be2ced817d66, kind: Service { id: d2bdfdcf-c904-463a-87e2-60b209fd743f }, name: Name("nexus-d2bdfdcf-c904-463a-87e2-60b209fd743f"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 186, 218])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f830fe44-0004-4968-88e2-304301a5a810, kind: Service { id: 060b7a36-3ed1-4e47-89cf-3f46bfb09b6c }, name: Name("nexus-060b7a36-3ed1-4e47-89cf-3f46bfb09b6c"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 157, 118])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
16332024-11-02T03:57:55.859ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
16342024-11-02T03:57:55.860ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
16352024-11-02T03:57:55.860ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
16362024-11-02T03:57:55.860ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte0" rules = []
16372024-11-02T03:57:55.860ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
16382024-11-02T03:57:55.860ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
16392024-11-02T03:57:55.860ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 4517 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:48591 req_id = ae8ac66e-6d3f-4f58-b463-591fc95c971b response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
16402024-11-02T03:57:55.862ZINFOSledAgent (RSS): Handoff to Nexus is complete file = sled-agent/src/rack_setup/service.rs:968
16412024-11-02T03:57:56.231ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:41783
16422024-11-02T03:57:56.231ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
16432024-11-02T03:57:56.231ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
16442024-11-02T03:57:56.231ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/6308f1ec-c08e-4282-a1c2-db37d6b8a044/config/omicron-zones.json file = common/src/ledger.rs:177
16452024-11-02T03:57:56.231ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/fda73319-9c6d-44d8-9377-b69537374c59/config/omicron-zones.json file = common/src/ledger.rs:177
16462024-11-02T03:57:56.413ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 182070 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:41783 req_id = 1383fc47-f25e-471c-bea1-7a992c6913c5 response_code = 200 uri = /inventory
16472024-11-02T03:57:57.322ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:64997
16482024-11-02T03:57:57.323ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:731 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 27d7c9ec-b8a8-4b31-ae12-c49ad36788f1, kind: Service { id: 7f79c18f-7b53-4b8a-86fc-bb23b923ea1d }, name: Name("external-dns-7f79c18f-7b53-4b8a-86fc-bb23b923ea1d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 238, 20])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 29b92946-7588-4b89-8147-322049e1f231, kind: Service { id: f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b }, name: Name("external-dns-f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 239, 77])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0cd6826d-84a0-4cd5-b7c6-32cb83627e6c, kind: Service { id: 9325209d-81b8-49e4-be6a-cedff3ef78c1 }, name: Name("nexus-9325209d-81b8-49e4-be6a-cedff3ef78c1"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 192, 104])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ec778b7a-3947-4776-98f2-be2ced817d66, kind: Service { id: d2bdfdcf-c904-463a-87e2-60b209fd743f }, name: Name("nexus-d2bdfdcf-c904-463a-87e2-60b209fd743f"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 186, 218])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f830fe44-0004-4968-88e2-304301a5a810, kind: Service { id: 060b7a36-3ed1-4e47-89cf-3f46bfb09b6c }, name: Name("nexus-060b7a36-3ed1-4e47-89cf-3f46bfb09b6c"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 157, 118])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
16492024-11-02T03:57:57.323ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
16502024-11-02T03:57:57.323ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
16512024-11-02T03:57:57.323ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
16522024-11-02T03:57:57.323ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte0" rules = []
16532024-11-02T03:57:57.323ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
16542024-11-02T03:57:57.323ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
16552024-11-02T03:57:57.323ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 357 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:64997 req_id = e21cd634-9cdc-4005-b250-8021bff038db response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
16562024-11-02T03:57:57.729ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:36598
16572024-11-02T03:57:57.729ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:731 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 27d7c9ec-b8a8-4b31-ae12-c49ad36788f1, kind: Service { id: 7f79c18f-7b53-4b8a-86fc-bb23b923ea1d }, name: Name("external-dns-7f79c18f-7b53-4b8a-86fc-bb23b923ea1d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 238, 20])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 29b92946-7588-4b89-8147-322049e1f231, kind: Service { id: f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b }, name: Name("external-dns-f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 239, 77])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0cd6826d-84a0-4cd5-b7c6-32cb83627e6c, kind: Service { id: 9325209d-81b8-49e4-be6a-cedff3ef78c1 }, name: Name("nexus-9325209d-81b8-49e4-be6a-cedff3ef78c1"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 192, 104])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ec778b7a-3947-4776-98f2-be2ced817d66, kind: Service { id: d2bdfdcf-c904-463a-87e2-60b209fd743f }, name: Name("nexus-d2bdfdcf-c904-463a-87e2-60b209fd743f"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 186, 218])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f830fe44-0004-4968-88e2-304301a5a810, kind: Service { id: 060b7a36-3ed1-4e47-89cf-3f46bfb09b6c }, name: Name("nexus-060b7a36-3ed1-4e47-89cf-3f46bfb09b6c"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 157, 118])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
16582024-11-02T03:57:57.729ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
16592024-11-02T03:57:57.729ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
16602024-11-02T03:57:57.729ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
16612024-11-02T03:57:57.729ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte0" rules = []
16622024-11-02T03:57:57.729ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
16632024-11-02T03:57:57.729ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
16642024-11-02T03:57:57.729ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 313 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:36598 req_id = e0046fc7-34b6-4be3-baa3-e3f061b4e938 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
16652024-11-02T03:57:57.764ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:46525
16662024-11-02T03:57:57.764ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:731 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 27d7c9ec-b8a8-4b31-ae12-c49ad36788f1, kind: Service { id: 7f79c18f-7b53-4b8a-86fc-bb23b923ea1d }, name: Name("external-dns-7f79c18f-7b53-4b8a-86fc-bb23b923ea1d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 238, 20])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 29b92946-7588-4b89-8147-322049e1f231, kind: Service { id: f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b }, name: Name("external-dns-f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 239, 77])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0cd6826d-84a0-4cd5-b7c6-32cb83627e6c, kind: Service { id: 9325209d-81b8-49e4-be6a-cedff3ef78c1 }, name: Name("nexus-9325209d-81b8-49e4-be6a-cedff3ef78c1"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 192, 104])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ec778b7a-3947-4776-98f2-be2ced817d66, kind: Service { id: d2bdfdcf-c904-463a-87e2-60b209fd743f }, name: Name("nexus-d2bdfdcf-c904-463a-87e2-60b209fd743f"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 186, 218])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f830fe44-0004-4968-88e2-304301a5a810, kind: Service { id: 060b7a36-3ed1-4e47-89cf-3f46bfb09b6c }, name: Name("nexus-060b7a36-3ed1-4e47-89cf-3f46bfb09b6c"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 157, 118])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
16672024-11-02T03:57:57.764ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
16682024-11-02T03:57:57.764ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
16692024-11-02T03:57:57.764ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
16702024-11-02T03:57:57.764ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte0" rules = []
16712024-11-02T03:57:57.764ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
16722024-11-02T03:57:57.764ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
16732024-11-02T03:57:57.764ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 184 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:46525 req_id = e87891f0-1c2a-4eb5-a072-910ce8a7a2e8 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
16742024-11-02T03:57:59.717ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:43620
16752024-11-02T03:57:59.717ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 53 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:43620 req_id = 7f562c70-fb02-4c94-87ec-a211e251760b response_code = 200 uri = /baseboard
16762024-11-02T03:58:01.598ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:40884
16772024-11-02T03:58:01.599ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 977 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:40884 req_id = 8747d5c6-5e46-4d57-8453-03604a432946 response_code = 200 uri = /vpc-routes
16782024-11-02T03:58:01.624ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
16792024-11-02T03:58:01.624ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 123 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:40884 req_id = 877cf6c8-93c3-431a-8f60-1d1d5d8d3175 response_code = 204 uri = /eip-gateways
16802024-11-02T03:58:01.714ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 3349 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:40884 req_id = 8bab1da9-0a79-4e99-b38f-c0aecb0a3ba9 response_code = 204 uri = /vpc-routes
16812024-11-02T03:58:12.836ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:47689
16822024-11-02T03:58:12.836ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 96 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:47689 req_id = 15192f21-affb-4735-92d1-acddadcfddb5 response_code = 200 uri = /vpc-routes
16832024-11-02T03:58:12.901ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
16842024-11-02T03:58:12.901ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 106 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:47689 req_id = 9b3975e3-9983-4690-a411-b9dc172c88f7 response_code = 204 uri = /eip-gateways
16852024-11-02T03:58:13.115ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 194 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:47689 req_id = 3e97d117-9176-4212-963b-aab82980bfa8 response_code = 204 uri = /vpc-routes
16862024-11-02T03:58:13.240ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:55035
16872024-11-02T03:58:13.240ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 267 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:55035 req_id = 935db2f3-dabc-4593-83c4-a23883f2ef43 response_code = 200 uri = /v2p
16882024-11-02T03:58:13.331ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:49188
16892024-11-02T03:58:13.332ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
16902024-11-02T03:58:13.415ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
16912024-11-02T03:58:13.496ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 165129 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:49188 req_id = 8c60d3ae-c4f4-4f64-a0ef-e6277649d228 response_code = 204 uri = /switch-ports
16922024-11-02T03:58:13.501ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 4543 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:49188 req_id = a19ec757-1474-4f29-8528-650cf3b2b92e response_code = 200 uri = /network-bootstore-config
16932024-11-02T03:58:13.550ZINFOSledAgent (bootstore): Attempting to update network config with generation: 2, current_generation: 1 file = bootstore/src/schemes/v0/peer.rs:564 peer_id = pc-buskin-i86pc
16942024-11-02T03:58:13.551ZINFOSledAgent (bootstore): Writing ledger to /pool/int/6308f1ec-c08e-4282-a1c2-db37d6b8a044/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-buskin-i86pc
16952024-11-02T03:58:13.552ZINFOSledAgent (bootstore): Writing ledger to /pool/int/fda73319-9c6d-44d8-9377-b69537374c59/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-buskin-i86pc
16962024-11-02T03:58:13.554ZINFOSledAgent (bootstore): Broadcasting network config with generation 2 file = bootstore/src/schemes/v0/peer.rs:648 peer_id = pc-buskin-i86pc
16972024-11-02T03:58:13.554ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 5993 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:49188 req_id = c3ff80db-1e81-45e4-84c5-51eb75c1d154 response_code = 204 uri = /network-bootstore-config
16982024-11-02T03:58:13.732ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:38214
16992024-11-02T03:58:13.732ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 63 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:38214 req_id = 13aa20e0-5939-4e19-bc38-4adc4a218310 response_code = 200 uri = /vpc-routes
17002024-11-02T03:58:13.779ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
17012024-11-02T03:58:13.779ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 67 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:38214 req_id = 73825b10-dcdc-4e8b-a8e7-9cf0683ef385 response_code = 204 uri = /eip-gateways
17022024-11-02T03:58:13.842ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:47193
17032024-11-02T03:58:13.843ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 154 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:47193 req_id = a2daa814-97c9-4adc-bfc2-a328bfe94c3e response_code = 200 uri = /v2p
17042024-11-02T03:58:13.903ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:60117
17052024-11-02T03:58:13.903ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 136 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:60117 req_id = e4613f10-62b6-45e5-b45f-238ba10fd2a2 response_code = 200 uri = /v2p
17062024-11-02T03:58:14.006ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:58678
17072024-11-02T03:58:14.006ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 56 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:58678 req_id = 40076645-cd89-43dd-ba7f-e53f874ac20c response_code = 200 uri = /vpc-routes
17082024-11-02T03:58:14.025ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
17092024-11-02T03:58:14.025ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 169 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:58678 req_id = ebf67e3e-6969-42e5-a9e8-676773863841 response_code = 204 uri = /eip-gateways
17102024-11-02T03:58:14.025ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 283 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:38214 req_id = da268488-26ea-41cc-b8ed-be23f6b336eb response_code = 204 uri = /vpc-routes
17112024-11-02T03:58:14.035ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 115 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:58678 req_id = a4aaa29b-a907-43f2-af22-ba209339ffd6 response_code = 204 uri = /vpc-routes
17122024-11-02T03:58:14.144ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:46546
17132024-11-02T03:58:14.144ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
17142024-11-02T03:58:14.187ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
17152024-11-02T03:58:14.244ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 99329 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:46546 req_id = 23ff6f17-135a-45b2-aa0f-a5c9a02419c7 response_code = 204 uri = /switch-ports
17162024-11-02T03:58:14.244ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:62767
17172024-11-02T03:58:14.244ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
17182024-11-02T03:58:14.281ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
17192024-11-02T03:58:14.335ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 91290 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:62767 req_id = 7e04a000-23d1-456d-8cad-b9414380e3f9 response_code = 204 uri = /switch-ports
17202024-11-02T03:58:14.336ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 270 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:62767 req_id = 6ae9ca9b-7140-4f7c-a7cb-3dca5ba9eab5 response_code = 200 uri = /network-bootstore-config
17212024-11-02T03:58:14.336ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 366 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:46546 req_id = 0c94c42c-c759-4f69-89bc-81fb4d917ce8 response_code = 200 uri = /network-bootstore-config
17222024-11-02T03:58:29.751ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:54647
17232024-11-02T03:58:29.752ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 30 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:54647 req_id = 6423ca49-a3d0-4ef8-9b4a-c60da7e1fa95 response_code = 200 uri = /baseboard
17242024-11-02T03:58:42.846ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:35315
17252024-11-02T03:58:42.846ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 84 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:35315 req_id = 72c2ac91-1879-47cf-b090-18b5640e56dd response_code = 200 uri = /vpc-routes
17262024-11-02T03:58:42.879ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
17272024-11-02T03:58:42.879ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 70 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:35315 req_id = 367daabc-3c8f-4365-b8ae-e3f4b441b303 response_code = 204 uri = /eip-gateways
17282024-11-02T03:58:42.960ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:56994
17292024-11-02T03:58:42.960ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 131 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:56994 req_id = e47b8b5d-295d-4fb7-871b-f37f4bd9c159 response_code = 200 uri = /v2p
17302024-11-02T03:58:42.969ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 180 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:35315 req_id = 6aa4234d-0907-419f-ae3a-1f8d10e6ffb9 response_code = 204 uri = /vpc-routes
17312024-11-02T03:58:43.174ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:65412
17322024-11-02T03:58:43.174ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
17332024-11-02T03:58:43.208ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
17342024-11-02T03:58:43.253ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 79180 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:65412 req_id = 1a9189d3-5b79-4973-a668-98c6aabf2fbe response_code = 204 uri = /switch-ports
17352024-11-02T03:58:43.254ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 112 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:65412 req_id = 67993148-1561-4beb-b970-c0fd28ecc152 response_code = 200 uri = /network-bootstore-config
17362024-11-02T03:58:43.647ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:48059
17372024-11-02T03:58:43.647ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 84 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:48059 req_id = f2415034-7e0a-4032-8b76-b6cf796e52a3 response_code = 200 uri = /vpc-routes
17382024-11-02T03:58:43.709ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
17392024-11-02T03:58:43.709ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 86 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:48059 req_id = 065b2399-e8c1-4039-bb39-d189a5c955c6 response_code = 204 uri = /eip-gateways
17402024-11-02T03:58:43.718ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 185 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:48059 req_id = 7c49ad05-9226-47e2-a5a0-fbdec44fdec4 response_code = 204 uri = /vpc-routes
17412024-11-02T03:58:43.726ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:35101
17422024-11-02T03:58:43.726ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 116 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:35101 req_id = d061e3d8-4af4-4446-8cf6-414e08bb9e4b response_code = 200 uri = /v2p
17432024-11-02T03:58:43.750ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:44623
17442024-11-02T03:58:43.750ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 25 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:44623 req_id = 3b7b946e-67af-4c88-b28d-f36c4c542245 response_code = 200 uri = /vpc-routes
17452024-11-02T03:58:43.759ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:41652
17462024-11-02T03:58:43.759ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
17472024-11-02T03:58:43.795ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
17482024-11-02T03:58:43.848ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 88352 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:41652 req_id = 83a9c5cd-8834-4e46-9d41-b6faf2c76adf response_code = 204 uri = /switch-ports
17492024-11-02T03:58:43.848ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:50513
17502024-11-02T03:58:43.848ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
17512024-11-02T03:58:43.848ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 104 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:44623 req_id = 91cabbd3-f1f6-44fd-9c98-d37fc232b782 response_code = 204 uri = /eip-gateways
17522024-11-02T03:58:43.848ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 220 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:41652 req_id = 5ffbb802-3bba-47aa-9953-d93ba4003fff response_code = 200 uri = /network-bootstore-config
17532024-11-02T03:58:43.848ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 250 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:50513 req_id = ff5aa8a7-eed8-41e8-a8f3-2290972eefc6 response_code = 200 uri = /v2p
17542024-11-02T03:58:43.858ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 156 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:44623 req_id = cc81b01d-ee25-445b-a98d-e3231c8fcd2c response_code = 204 uri = /vpc-routes
17552024-11-02T03:58:43.972ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:61771
17562024-11-02T03:58:43.972ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
17572024-11-02T03:58:44.010ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
17582024-11-02T03:58:44.069ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 96790 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:61771 req_id = 78ac7cc8-8bab-41ae-a634-4314050fe238 response_code = 204 uri = /switch-ports
17592024-11-02T03:58:44.069ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 110 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:61771 req_id = 6095e0d4-9837-49e9-bacd-04f205d051e0 response_code = 200 uri = /network-bootstore-config
17602024-11-02T03:58:59.789ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:39621
17612024-11-02T03:58:59.789ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 56 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:39621 req_id = a5636d55-51b4-4750-a79e-ccaeea83a923 response_code = 200 uri = /baseboard
17622024-11-02T03:59:12.979ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:48567
17632024-11-02T03:59:12.979ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 74 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:48567 req_id = 6bf270b5-e3a6-4143-870e-408ad0b179f6 response_code = 200 uri = /vpc-routes
17642024-11-02T03:59:12.997ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
17652024-11-02T03:59:12.997ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 61 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:48567 req_id = 2b48a224-9de9-486a-b118-012832271fed response_code = 204 uri = /eip-gateways
17662024-11-02T03:59:13.030ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:33271
17672024-11-02T03:59:13.030ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 142 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:33271 req_id = d56ff68b-1080-4925-92e3-f6697af6fab5 response_code = 200 uri = /v2p
17682024-11-02T03:59:13.036ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 170 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:48567 req_id = 3332afc1-cdad-43be-b269-5d02ef1a145b response_code = 204 uri = /vpc-routes
17692024-11-02T03:59:13.284ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:43018
17702024-11-02T03:59:13.284ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
17712024-11-02T03:59:13.322ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
17722024-11-02T03:59:13.377ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 92381 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:43018 req_id = 4e1a0f0c-bf27-4cf4-8177-94bc28014381 response_code = 204 uri = /switch-ports
17732024-11-02T03:59:13.377ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 100 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:43018 req_id = 82f86905-823e-4c46-afa4-654858b3528b response_code = 200 uri = /network-bootstore-config
17742024-11-02T03:59:13.781ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:49111
17752024-11-02T03:59:13.781ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 85 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:49111 req_id = 4c4b19f6-de94-42ef-a751-ae9bf7e89271 response_code = 200 uri = /vpc-routes
17762024-11-02T03:59:13.787ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:42382
17772024-11-02T03:59:13.787ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 174 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:42382 req_id = 4d57422e-b6e6-48e0-8142-11f1dea7d013 response_code = 200 uri = /v2p
17782024-11-02T03:59:13.793ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:59617
17792024-11-02T03:59:13.793ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 36 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:59617 req_id = 9767ea27-e80e-4d59-9431-f47c83718e38 response_code = 200 uri = /vpc-routes
17802024-11-02T03:59:13.846ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
17812024-11-02T03:59:13.846ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 114 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:59617 req_id = cc28be5a-a24c-4dc3-a1fc-0e73ee197547 response_code = 204 uri = /eip-gateways
17822024-11-02T03:59:13.846ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
17832024-11-02T03:59:13.846ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 46 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:49111 req_id = e27390f5-2afa-4a83-bbe9-2a2ee48031c6 response_code = 204 uri = /eip-gateways
17842024-11-02T03:59:13.855ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 189 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:59617 req_id = 1c04f891-fedc-402f-ae11-2051959203a8 response_code = 204 uri = /vpc-routes
17852024-11-02T03:59:13.866ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 130 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:49111 req_id = eeb0f408-ddf7-4b9b-846d-80bca1e81008 response_code = 204 uri = /vpc-routes
17862024-11-02T03:59:13.979ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:38256
17872024-11-02T03:59:13.979ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
17882024-11-02T03:59:14.017ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
17892024-11-02T03:59:14.076ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 97212 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:38256 req_id = 3dde85f6-9590-4f06-864d-676df2ff5b64 response_code = 204 uri = /switch-ports
17902024-11-02T03:59:14.076ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:43347
17912024-11-02T03:59:14.077ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 113 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:43347 req_id = 8678a433-ab53-4b77-a33c-e1cf6527b434 response_code = 200 uri = /v2p
17922024-11-02T03:59:14.077ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 83 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:38256 req_id = 28ff9744-014c-4e39-ab80-b7f55f3990e3 response_code = 200 uri = /network-bootstore-config
17932024-11-02T03:59:14.093ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:62385
17942024-11-02T03:59:14.093ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
17952024-11-02T03:59:14.128ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
17962024-11-02T03:59:14.182ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 89022 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:62385 req_id = c37b7f19-0fc2-4d79-81af-86a6eade1410 response_code = 204 uri = /switch-ports
17972024-11-02T03:59:14.183ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 147 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:62385 req_id = 604a4c1a-2b28-4400-af35-d796239eea26 response_code = 200 uri = /network-bootstore-config
17982024-11-02T03:59:29.825ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:47184
17992024-11-02T03:59:29.826ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 74 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:47184 req_id = 80365f11-eaa6-4ca8-bd38-f96325062d88 response_code = 200 uri = /baseboard
18002024-11-02T03:59:42.783ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:36841
18012024-11-02T03:59:42.784ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 71 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:36841 req_id = 6a7a01cd-cb77-4910-9f8f-871aaa3dfa4b response_code = 200 uri = /vpc-routes
18022024-11-02T03:59:42.829ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
18032024-11-02T03:59:42.829ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 77 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:36841 req_id = 0a520980-b677-4f2f-9f1c-c8d77f4a2a41 response_code = 204 uri = /eip-gateways
18042024-11-02T03:59:42.914ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:56765
18052024-11-02T03:59:42.914ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 179 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:56765 req_id = 24c1dce5-da41-487d-b959-14f789aeae16 response_code = 200 uri = /v2p
18062024-11-02T03:59:42.923ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 145 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:36841 req_id = fdea8f43-99dc-4b7c-a3d2-c428af7ba30c response_code = 204 uri = /vpc-routes
18072024-11-02T03:59:43.188ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:43472
18082024-11-02T03:59:43.188ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
18092024-11-02T03:59:43.225ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
18102024-11-02T03:59:43.278ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 89457 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:43472 req_id = eca1af20-8233-401f-9b8f-6e6706723a3b response_code = 204 uri = /switch-ports
18112024-11-02T03:59:43.278ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 98 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:43472 req_id = afb9ab9b-d463-4279-befe-429bc2441f20 response_code = 200 uri = /network-bootstore-config
18122024-11-02T03:59:43.660ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:53356
18132024-11-02T03:59:43.660ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 73 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:53356 req_id = 98d17bd2-8bff-446f-8463-dfec5991f4bf response_code = 200 uri = /vpc-routes
18142024-11-02T03:59:43.677ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
18152024-11-02T03:59:43.677ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 80 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:53356 req_id = 71f9aae3-0292-49dd-9616-fdde381edb5d response_code = 204 uri = /eip-gateways
18162024-11-02T03:59:43.689ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 197 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:53356 req_id = 61cf32c5-92f3-4e66-8561-cb936d7f1644 response_code = 204 uri = /vpc-routes
18172024-11-02T03:59:43.751ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51440
18182024-11-02T03:59:43.751ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 46 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:51440 req_id = 1eef671f-e2a1-45c6-a5cb-08a1aa7518ed response_code = 200 uri = /vpc-routes
18192024-11-02T03:59:43.757ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:52524
18202024-11-02T03:59:43.758ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 142 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:52524 req_id = 1396f870-ab92-4c75-bfb6-2ecf4b8001cf response_code = 200 uri = /v2p
18212024-11-02T03:59:43.772ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:59225
18222024-11-02T03:59:43.772ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
18232024-11-02T03:59:43.806ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
18242024-11-02T03:59:43.860ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 87864 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:59225 req_id = 985d87ae-3a44-4aa4-aa24-58834400fd66 response_code = 204 uri = /switch-ports
18252024-11-02T03:59:43.860ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51160
18262024-11-02T03:59:43.860ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:62948
18272024-11-02T03:59:43.860ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
18282024-11-02T03:59:43.861ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 168 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:51440 req_id = 4d632202-8985-45a2-84a9-acd7cd1dd348 response_code = 204 uri = /eip-gateways
18292024-11-02T03:59:43.861ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 206 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:51160 req_id = 909a2ff7-4537-4453-9c36-4603c179f8f4 response_code = 200 uri = /v2p
18302024-11-02T03:59:43.861ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 290 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:59225 req_id = 2b512c0e-33f1-418c-8d12-ad1feb148fd0 response_code = 200 uri = /network-bootstore-config
18312024-11-02T03:59:43.861ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
18322024-11-02T03:59:43.872ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 238 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:51440 req_id = fd834fc5-1a01-4abb-ac70-874283bbb82c response_code = 204 uri = /vpc-routes
18332024-11-02T03:59:43.902ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
18342024-11-02T03:59:43.960ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 99301 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:62948 req_id = 1c8af847-38da-41d7-babb-8e188849bfc3 response_code = 204 uri = /switch-ports
18352024-11-02T03:59:43.961ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 215 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:62948 req_id = 5b975002-6778-41a9-8003-0e7261908cc0 response_code = 200 uri = /network-bootstore-config
18362024-11-02T03:59:59.859ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:56552
18372024-11-02T03:59:59.859ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 28 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:56552 req_id = 9321ec91-66c8-40f0-828d-06576b5beb00 response_code = 200 uri = /baseboard
18382024-11-02T04:00:12.962ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:37593
18392024-11-02T04:00:12.962ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 61 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:37593 req_id = defa3dcb-f507-45d9-aa54-686f0579bb00 response_code = 200 uri = /vpc-routes
18402024-11-02T04:00:13.004ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
18412024-11-02T04:00:13.004ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 92 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:37593 req_id = 90881c9a-1944-4338-85bc-d2c0ef565b1a response_code = 204 uri = /eip-gateways
18422024-11-02T04:00:13.014ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 191 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:37593 req_id = 82505573-bb15-4ed9-a61c-7f15177d459f response_code = 204 uri = /vpc-routes
18432024-11-02T04:00:13.127ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:54754
18442024-11-02T04:00:13.127ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 158 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:54754 req_id = ee8ca00f-b039-40c3-9435-53c5224f4576 response_code = 200 uri = /v2p
18452024-11-02T04:00:13.179ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:44210
18462024-11-02T04:00:13.179ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
18472024-11-02T04:00:13.216ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
18482024-11-02T04:00:13.269ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 90213 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:44210 req_id = 61f94daa-4bc0-4ab1-9ad2-4f9b74b64251 response_code = 204 uri = /switch-ports
18492024-11-02T04:00:13.270ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 115 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:44210 req_id = 761f692e-c74f-4697-961b-c4048959dfcf response_code = 200 uri = /network-bootstore-config
18502024-11-02T04:00:13.736ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:48422
18512024-11-02T04:00:13.736ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 105 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:48422 req_id = 434b2f7b-d07c-43d0-90f6-c68686a2bc7a response_code = 200 uri = /vpc-routes
18522024-11-02T04:00:13.751ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
18532024-11-02T04:00:13.751ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 87 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:48422 req_id = 165c88fc-83c1-47e2-b30e-21af9dcb8dc4 response_code = 204 uri = /eip-gateways
18542024-11-02T04:00:13.761ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 261 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:48422 req_id = b3ad946b-ea37-4022-bf6a-8cb6b50b7f8d response_code = 204 uri = /vpc-routes
18552024-11-02T04:00:13.900ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:36354
18562024-11-02T04:00:13.900ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 161 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:36354 req_id = 134d9f25-92b8-4cbd-8ac4-5c08ec6ac66f response_code = 200 uri = /v2p
18572024-11-02T04:00:13.976ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:59765
18582024-11-02T04:00:13.976ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 141 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:59765 req_id = a997b7fc-88b4-49a8-912f-ae695280e919 response_code = 200 uri = /v2p
18592024-11-02T04:00:14.019ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:45533
18602024-11-02T04:00:14.019ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 69 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:45533 req_id = 84dfcb59-e44c-492a-910d-b921acdb3477 response_code = 200 uri = /vpc-routes
18612024-11-02T04:00:14.049ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:53032
18622024-11-02T04:00:14.050ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
18632024-11-02T04:00:14.091ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
18642024-11-02T04:00:14.148ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 98018 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:53032 req_id = 0d3751a4-4880-47fc-8ac9-942072f36cec response_code = 204 uri = /switch-ports
18652024-11-02T04:00:14.148ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:39738
18662024-11-02T04:00:14.148ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
18672024-11-02T04:00:14.148ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 148 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:45533 req_id = 68369a82-fb9c-425c-84dd-03116cf9f15a response_code = 204 uri = /eip-gateways
18682024-11-02T04:00:14.148ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
18692024-11-02T04:00:14.148ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 264 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:53032 req_id = eb3c9b9a-2e6b-4553-a000-21275a788761 response_code = 200 uri = /network-bootstore-config
18702024-11-02T04:00:14.157ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 373 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:45533 req_id = 5f7253e3-3b44-4aca-aa82-27e8072b102a response_code = 204 uri = /vpc-routes
18712024-11-02T04:00:14.183ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
18722024-11-02T04:00:14.235ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 87298 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:39738 req_id = 095b3cfc-4569-4e40-ab65-4fdec566f70c response_code = 204 uri = /switch-ports
18732024-11-02T04:00:14.236ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 140 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:39738 req_id = 9ad2347a-7bfc-48d4-8143-6abd34dd688b response_code = 200 uri = /network-bootstore-config
18742024-11-02T04:00:23.851ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
18752024-11-02T04:00:29.893ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:36949
18762024-11-02T04:00:29.894ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 168 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:36949 req_id = 579a573d-70f0-43ee-929c-50e2cc78a8f3 response_code = 200 uri = /baseboard
18772024-11-02T04:00:42.875ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:36044
18782024-11-02T04:00:42.875ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 56 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:36044 req_id = 0db5f4aa-0194-4ab0-953d-a0cf92375fe3 response_code = 200 uri = /vpc-routes
18792024-11-02T04:00:42.923ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
18802024-11-02T04:00:42.923ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 78 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:36044 req_id = 09a27055-d7e2-42d8-a76b-6f11161f6e9c response_code = 204 uri = /eip-gateways
18812024-11-02T04:00:42.932ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 164 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:36044 req_id = 6c4f2729-4236-46a4-82da-c899808ea35e response_code = 204 uri = /vpc-routes
18822024-11-02T04:00:42.968ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:55488
18832024-11-02T04:00:42.968ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 104 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:55488 req_id = 6321b42f-d959-48c3-ac36-9cbf4d295530 response_code = 200 uri = /v2p
18842024-11-02T04:00:43.008ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:61266
18852024-11-02T04:00:43.009ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
18862024-11-02T04:00:43.048ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
18872024-11-02T04:00:43.104ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 95024 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:61266 req_id = 696d9dc6-fda7-43a0-bd77-b00ce3ef4c5e response_code = 204 uri = /switch-ports
18882024-11-02T04:00:43.104ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 120 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:61266 req_id = b98ecf37-519e-4764-b8c9-d9b5b7a55dcf response_code = 200 uri = /network-bootstore-config
18892024-11-02T04:00:43.571ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:64255
18902024-11-02T04:00:43.571ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 99 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:64255 req_id = 20734034-931a-4e1d-b2ea-a51d323fb6c3 response_code = 200 uri = /vpc-routes
18912024-11-02T04:00:43.622ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
18922024-11-02T04:00:43.622ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 96 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:64255 req_id = b68e2e5b-0e97-4617-83c3-1a923fe6ae49 response_code = 204 uri = /eip-gateways
18932024-11-02T04:00:43.644ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 242 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:64255 req_id = 091f9adb-17ee-424c-952b-3323ed79cd60 response_code = 204 uri = /vpc-routes
18942024-11-02T04:00:43.712ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:43679
18952024-11-02T04:00:43.712ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 161 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:43679 req_id = d3d91265-aee1-4ad9-b33a-04ca48afff77 response_code = 200 uri = /v2p
18962024-11-02T04:00:43.744ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:38174
18972024-11-02T04:00:43.745ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
18982024-11-02T04:00:43.788ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
18992024-11-02T04:00:43.844ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 99171 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:38174 req_id = 8ae00cf1-f5ef-4dae-a8a4-d4f98081c114 response_code = 204 uri = /switch-ports
19002024-11-02T04:00:43.844ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:53554
19012024-11-02T04:00:43.844ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 54 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:53554 req_id = f9194b1d-43f5-475e-b0bd-bff986245690 response_code = 200 uri = /vpc-routes
19022024-11-02T04:00:43.844ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 89 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:38174 req_id = d144ab9a-9e5a-4c93-8f09-a3a81697eabf response_code = 200 uri = /network-bootstore-config
19032024-11-02T04:00:43.891ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
19042024-11-02T04:00:43.891ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 110 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:53554 req_id = 014351af-8715-4211-b91f-a086f1e2355a response_code = 204 uri = /eip-gateways
19052024-11-02T04:00:43.907ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:50518
19062024-11-02T04:00:43.907ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 126 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:50518 req_id = 66a037fc-4198-4b1e-9aea-735256f4fac4 response_code = 200 uri = /v2p
19072024-11-02T04:00:43.913ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 187 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:53554 req_id = a41b25e5-b120-4523-af28-037db63d2b16 response_code = 204 uri = /vpc-routes
19082024-11-02T04:00:43.930ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:49581
19092024-11-02T04:00:43.930ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
19102024-11-02T04:00:43.969ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
19112024-11-02T04:00:44.026ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 95638 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:49581 req_id = 9e382a05-e839-4ea1-bc47-aa01d952a3ff response_code = 204 uri = /switch-ports
19122024-11-02T04:00:44.026ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 149 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:49581 req_id = ef8f96e7-0e05-4d7d-91a3-39c5ce079ce5 response_code = 200 uri = /network-bootstore-config
19132024-11-02T04:00:59.926ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:43473
19142024-11-02T04:00:59.926ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 29 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:43473 req_id = 539dc3ea-8980-4750-84e3-08ca913a5509 response_code = 200 uri = /baseboard
19152024-11-02T04:01:12.960ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:56732
19162024-11-02T04:01:12.961ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 89 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:56732 req_id = 3c21fcad-2b56-4bc4-8e03-8525415dec92 response_code = 200 uri = /vpc-routes
19172024-11-02T04:01:13.002ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
19182024-11-02T04:01:13.002ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 72 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:56732 req_id = cca4ea0c-fd39-44e2-a5b9-713fed759ce2 response_code = 204 uri = /eip-gateways
19192024-11-02T04:01:13.005ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:64963
19202024-11-02T04:01:13.006ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 129 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:64963 req_id = 037374e5-7d19-4396-9fcd-b15ee40943c1 response_code = 200 uri = /v2p
19212024-11-02T04:01:13.010ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 174 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:56732 req_id = b9a565a3-bff5-42b4-be18-2b03e2498b07 response_code = 204 uri = /vpc-routes
19222024-11-02T04:01:13.161ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:62840
19232024-11-02T04:01:13.161ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
19242024-11-02T04:01:13.198ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
19252024-11-02T04:01:13.255ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 94130 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:62840 req_id = d215c0eb-cbaf-45f4-bcc7-e7874ac0581d response_code = 204 uri = /switch-ports
19262024-11-02T04:01:13.255ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 95 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:62840 req_id = a5a0cb7c-c911-4c15-b2ca-2b2df6a8ac75 response_code = 200 uri = /network-bootstore-config
19272024-11-02T04:01:13.868ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:58744
19282024-11-02T04:01:13.868ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 78 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:58744 req_id = c7c02dd8-0ea5-4b22-8751-5b97b7733e49 response_code = 200 uri = /vpc-routes
19292024-11-02T04:01:13.916ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:57473
19302024-11-02T04:01:13.916ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 70 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:57473 req_id = 3e963f43-4a96-40dc-be1d-8e3f7452b783 response_code = 200 uri = /vpc-routes
19312024-11-02T04:01:13.916ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
19322024-11-02T04:01:13.916ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 56 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:58744 req_id = 4dc4c9e4-bfef-42a9-aeeb-a1c4cd19a55b response_code = 204 uri = /eip-gateways
19332024-11-02T04:01:13.925ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 166 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:58744 req_id = 04013604-d601-4964-8170-5f745e17120e response_code = 204 uri = /vpc-routes
19342024-11-02T04:01:13.969ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
19352024-11-02T04:01:13.969ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 39 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:57473 req_id = 0c38c370-ca34-4c82-9f77-b9d310e5447d response_code = 204 uri = /eip-gateways
19362024-11-02T04:01:13.977ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 118 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:57473 req_id = 2ff53fae-9f21-4436-8c69-a1cde19e14e8 response_code = 204 uri = /vpc-routes
19372024-11-02T04:01:13.992ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:40431
19382024-11-02T04:01:13.992ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 100 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:40431 req_id = 1514c59f-aabd-453f-b6eb-b2099aef51b9 response_code = 200 uri = /v2p
19392024-11-02T04:01:13.992ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:59791
19402024-11-02T04:01:13.993ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 62 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:59791 req_id = 7bace770-0d18-49ae-9b4c-87264f60de46 response_code = 200 uri = /v2p
19412024-11-02T04:01:14.079ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:40886
19422024-11-02T04:01:14.079ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
19432024-11-02T04:01:14.117ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
19442024-11-02T04:01:14.174ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 94464 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:40886 req_id = 0877e569-3b9c-4f60-94e7-88fad759ec63 response_code = 204 uri = /switch-ports
19452024-11-02T04:01:14.174ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:52020
19462024-11-02T04:01:14.174ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
19472024-11-02T04:01:14.208ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
19482024-11-02T04:01:14.264ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 90045 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:52020 req_id = ddcad61c-0cf8-4400-8ecb-0d944e39437f response_code = 204 uri = /switch-ports
19492024-11-02T04:01:14.264ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 112 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:40886 req_id = de7b97b8-2430-4ff4-9d97-2168ffbf8ded response_code = 200 uri = /network-bootstore-config
19502024-11-02T04:01:14.264ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 185 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:52020 req_id = 7cc792ce-d361-4005-94d6-92297f209616 response_code = 200 uri = /network-bootstore-config
19512024-11-02T04:01:29.964ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:59826
19522024-11-02T04:01:29.964ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 38 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:59826 req_id = aa34721e-db41-4c47-9ab0-698dd1662cf1 response_code = 200 uri = /baseboard
19532024-11-02T04:01:42.809ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:48252
19542024-11-02T04:01:42.809ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 101 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:48252 req_id = 926b21fc-f683-4e77-8a38-640b1ac6fda8 response_code = 200 uri = /vpc-routes
19552024-11-02T04:01:42.827ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
19562024-11-02T04:01:42.827ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 176 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:48252 req_id = 4704a092-1de2-4bc3-b842-be342c49a51f response_code = 204 uri = /eip-gateways
19572024-11-02T04:01:42.835ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 171 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:48252 req_id = fc9046f0-96a3-412b-8537-e76a0bc4558d response_code = 204 uri = /vpc-routes
19582024-11-02T04:01:42.889ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:39649
19592024-11-02T04:01:42.889ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 138 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:39649 req_id = bfa02534-6600-4381-b393-087bbbf00236 response_code = 200 uri = /v2p
19602024-11-02T04:01:42.941ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:65101
19612024-11-02T04:01:42.941ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
19622024-11-02T04:01:42.979ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
19632024-11-02T04:01:43.032ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 90173 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:65101 req_id = b8eba42c-c60a-4383-8ce0-67881e7a0f6a response_code = 204 uri = /switch-ports
19642024-11-02T04:01:43.032ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 94 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:65101 req_id = ee4f1061-2a3b-4704-92e9-a1060760660e response_code = 200 uri = /network-bootstore-config
19652024-11-02T04:01:43.596ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:64579
19662024-11-02T04:01:43.596ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 76 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:64579 req_id = d4a420ae-bf81-46e0-83d6-599a76689c20 response_code = 200 uri = /vpc-routes
19672024-11-02T04:01:43.621ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
19682024-11-02T04:01:43.621ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 98 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:64579 req_id = 00a2ed9f-2746-4ab7-b087-3cb165e5ff20 response_code = 204 uri = /eip-gateways
19692024-11-02T04:01:43.631ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 200 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:64579 req_id = 8f44714f-e2a7-4d03-9e1e-06c9fb51126d response_code = 204 uri = /vpc-routes
19702024-11-02T04:01:43.767ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:55808
19712024-11-02T04:01:43.767ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 163 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:55808 req_id = d00d6846-5b16-4437-afc5-fee2cd90b198 response_code = 200 uri = /v2p
19722024-11-02T04:01:43.785ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:38169
19732024-11-02T04:01:43.785ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
19742024-11-02T04:01:43.824ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
19752024-11-02T04:01:43.880ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 95181 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:38169 req_id = e80cac02-963c-42ef-8988-68e1bea2fd81 response_code = 204 uri = /switch-ports
19762024-11-02T04:01:43.880ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:43962
19772024-11-02T04:01:43.880ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 63 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:43962 req_id = f3e85e4d-df7e-4436-bbd9-9dc8f4682041 response_code = 200 uri = /vpc-routes
19782024-11-02T04:01:43.880ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 105 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:38169 req_id = 9dc20790-9c4c-4ab7-925e-613634ca3a12 response_code = 200 uri = /network-bootstore-config
19792024-11-02T04:01:43.927ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
19802024-11-02T04:01:43.927ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 108 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:43962 req_id = 266cbe0e-4a3c-4453-97fa-9b64997d5a04 response_code = 204 uri = /eip-gateways
19812024-11-02T04:01:43.934ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51228
19822024-11-02T04:01:43.934ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 146 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:51228 req_id = 831e06ce-83e8-485f-befe-fd2c59edf73f response_code = 200 uri = /v2p
19832024-11-02T04:01:43.942ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 164 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:43962 req_id = e4fc2509-7758-483e-9c18-82e2a908c0a6 response_code = 204 uri = /vpc-routes
19842024-11-02T04:01:43.949ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:38034
19852024-11-02T04:01:43.949ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
19862024-11-02T04:01:43.987ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
19872024-11-02T04:01:44.042ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 92809 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:38034 req_id = b9fccc10-e716-4cc1-8537-b18663813585 response_code = 204 uri = /switch-ports
19882024-11-02T04:01:44.042ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 101 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:38034 req_id = 7bfcd151-328c-4dff-811f-896e758ff1cf response_code = 200 uri = /network-bootstore-config
19892024-11-02T04:02:00.002ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:61959
19902024-11-02T04:02:00.002ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 90 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:61959 req_id = b7ab90ee-77b6-491f-a9d5-bdccbc5e0148 response_code = 200 uri = /baseboard
19912024-11-02T04:02:13.086ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:34512
19922024-11-02T04:02:13.086ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 88 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:34512 req_id = c65afd42-774f-4ff5-a372-6efc47c59080 response_code = 200 uri = /vpc-routes
19932024-11-02T04:02:13.131ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
19942024-11-02T04:02:13.131ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 107 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:34512 req_id = 1b4b722d-38a4-4f5a-a3c8-d10f1824db07 response_code = 204 uri = /eip-gateways
19952024-11-02T04:02:13.140ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 172 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:34512 req_id = a6cec0d8-8a19-4c9c-8581-b40aae3800c3 response_code = 204 uri = /vpc-routes
19962024-11-02T04:02:13.220ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:43830
19972024-11-02T04:02:13.220ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 218 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:43830 req_id = fb6590f8-37ee-4a47-8b56-5d1430c79f9c response_code = 200 uri = /v2p
19982024-11-02T04:02:13.222ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:36947
19992024-11-02T04:02:13.222ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
20002024-11-02T04:02:13.262ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
20012024-11-02T04:02:13.318ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 95976 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:36947 req_id = 3964d56c-37ff-4afb-a988-2ee423f1b004 response_code = 204 uri = /switch-ports
20022024-11-02T04:02:13.319ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 149 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:36947 req_id = e1951e84-ce42-4648-961b-4b381b7813f8 response_code = 200 uri = /network-bootstore-config
20032024-11-02T04:02:13.396ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:51553
20042024-11-02T04:02:13.396ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:731 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 27d7c9ec-b8a8-4b31-ae12-c49ad36788f1, kind: Service { id: 7f79c18f-7b53-4b8a-86fc-bb23b923ea1d }, name: Name("external-dns-7f79c18f-7b53-4b8a-86fc-bb23b923ea1d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 238, 20])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 29b92946-7588-4b89-8147-322049e1f231, kind: Service { id: f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b }, name: Name("external-dns-f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 239, 77])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0cd6826d-84a0-4cd5-b7c6-32cb83627e6c, kind: Service { id: 9325209d-81b8-49e4-be6a-cedff3ef78c1 }, name: Name("nexus-9325209d-81b8-49e4-be6a-cedff3ef78c1"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 192, 104])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ec778b7a-3947-4776-98f2-be2ced817d66, kind: Service { id: d2bdfdcf-c904-463a-87e2-60b209fd743f }, name: Name("nexus-d2bdfdcf-c904-463a-87e2-60b209fd743f"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 186, 218])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f830fe44-0004-4968-88e2-304301a5a810, kind: Service { id: 060b7a36-3ed1-4e47-89cf-3f46bfb09b6c }, name: Name("nexus-060b7a36-3ed1-4e47-89cf-3f46bfb09b6c"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 157, 118])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
20052024-11-02T04:02:13.396ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
20062024-11-02T04:02:13.396ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
20072024-11-02T04:02:13.396ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
20082024-11-02T04:02:13.396ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte0" rules = []
20092024-11-02T04:02:13.396ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
20102024-11-02T04:02:13.396ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
20112024-11-02T04:02:13.396ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 412 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:51553 req_id = 204b2097-d97f-4fff-9362-a18b464f2ffe response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
20122024-11-02T04:02:13.783ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:34473
20132024-11-02T04:02:13.783ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 62 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:34473 req_id = 8b6e684c-4f65-49e9-b756-1a6982a73baf response_code = 200 uri = /vpc-routes
20142024-11-02T04:02:13.842ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
20152024-11-02T04:02:13.842ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 88 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:34473 req_id = 6807b9ad-658b-4636-9173-6e1b917422bb response_code = 204 uri = /eip-gateways
20162024-11-02T04:02:13.852ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 226 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:34473 req_id = 47a2be2a-1842-4f8f-a7b8-b8338dc51ef7 response_code = 204 uri = /vpc-routes
20172024-11-02T04:02:13.864ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:37429
20182024-11-02T04:02:13.864ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 41 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:37429 req_id = f377cfeb-48db-4195-b51a-244435d07533 response_code = 200 uri = /vpc-routes
20192024-11-02T04:02:13.922ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
20202024-11-02T04:02:13.923ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 64 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:37429 req_id = c27dfa2f-4f5e-4835-b4a4-93bccd67d899 response_code = 204 uri = /eip-gateways
20212024-11-02T04:02:13.931ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 197 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:37429 req_id = c017b18b-3991-4bd7-9a56-e2325d27dc57 response_code = 204 uri = /vpc-routes
20222024-11-02T04:02:14.013ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:47543
20232024-11-02T04:02:14.013ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 161 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:47543 req_id = a94d7cd4-0e26-4b22-aabd-4d214012d968 response_code = 200 uri = /v2p
20242024-11-02T04:02:14.052ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:42306
20252024-11-02T04:02:14.052ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
20262024-11-02T04:02:14.096ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
20272024-11-02T04:02:14.163ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 110922 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:42306 req_id = a56a8c56-6ba3-4cd5-8b59-a113ef3ee290 response_code = 204 uri = /switch-ports
20282024-11-02T04:02:14.163ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:52928
20292024-11-02T04:02:14.164ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 216 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:52928 req_id = 9ab50443-6749-4ce9-aedf-31e379730007 response_code = 200 uri = /v2p
20302024-11-02T04:02:14.164ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 276 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:42306 req_id = 1acebce1-a447-4b50-83c0-c4bfc4837989 response_code = 200 uri = /network-bootstore-config
20312024-11-02T04:02:14.174ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:64951
20322024-11-02T04:02:14.175ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:731 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 27d7c9ec-b8a8-4b31-ae12-c49ad36788f1, kind: Service { id: 7f79c18f-7b53-4b8a-86fc-bb23b923ea1d }, name: Name("external-dns-7f79c18f-7b53-4b8a-86fc-bb23b923ea1d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 238, 20])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 29b92946-7588-4b89-8147-322049e1f231, kind: Service { id: f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b }, name: Name("external-dns-f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 239, 77])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0cd6826d-84a0-4cd5-b7c6-32cb83627e6c, kind: Service { id: 9325209d-81b8-49e4-be6a-cedff3ef78c1 }, name: Name("nexus-9325209d-81b8-49e4-be6a-cedff3ef78c1"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 192, 104])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ec778b7a-3947-4776-98f2-be2ced817d66, kind: Service { id: d2bdfdcf-c904-463a-87e2-60b209fd743f }, name: Name("nexus-d2bdfdcf-c904-463a-87e2-60b209fd743f"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 186, 218])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f830fe44-0004-4968-88e2-304301a5a810, kind: Service { id: 060b7a36-3ed1-4e47-89cf-3f46bfb09b6c }, name: Name("nexus-060b7a36-3ed1-4e47-89cf-3f46bfb09b6c"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 157, 118])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
20332024-11-02T04:02:14.175ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
20342024-11-02T04:02:14.175ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
20352024-11-02T04:02:14.175ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
20362024-11-02T04:02:14.175ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte0" rules = []
20372024-11-02T04:02:14.175ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
20382024-11-02T04:02:14.175ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
20392024-11-02T04:02:14.175ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 461 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:64951 req_id = 5389cd5e-773e-464e-bff8-b39cdcb14950 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
20402024-11-02T04:02:14.227ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:47455
20412024-11-02T04:02:14.227ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
20422024-11-02T04:02:14.261ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
20432024-11-02T04:02:14.316ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 89153 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:47455 req_id = 06599639-dc6e-4dbb-b185-d450b9d8f3ab response_code = 204 uri = /switch-ports
20442024-11-02T04:02:14.316ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:43188
20452024-11-02T04:02:14.316ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:731 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 27d7c9ec-b8a8-4b31-ae12-c49ad36788f1, kind: Service { id: 7f79c18f-7b53-4b8a-86fc-bb23b923ea1d }, name: Name("external-dns-7f79c18f-7b53-4b8a-86fc-bb23b923ea1d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 238, 20])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 29b92946-7588-4b89-8147-322049e1f231, kind: Service { id: f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b }, name: Name("external-dns-f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 239, 77])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0cd6826d-84a0-4cd5-b7c6-32cb83627e6c, kind: Service { id: 9325209d-81b8-49e4-be6a-cedff3ef78c1 }, name: Name("nexus-9325209d-81b8-49e4-be6a-cedff3ef78c1"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 192, 104])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ec778b7a-3947-4776-98f2-be2ced817d66, kind: Service { id: d2bdfdcf-c904-463a-87e2-60b209fd743f }, name: Name("nexus-d2bdfdcf-c904-463a-87e2-60b209fd743f"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 186, 218])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f830fe44-0004-4968-88e2-304301a5a810, kind: Service { id: 060b7a36-3ed1-4e47-89cf-3f46bfb09b6c }, name: Name("nexus-060b7a36-3ed1-4e47-89cf-3f46bfb09b6c"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 157, 118])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
20462024-11-02T04:02:14.316ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 216 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:47455 req_id = 1021a2aa-6f2a-4c04-b856-b7d85c629a7e response_code = 200 uri = /network-bootstore-config
20472024-11-02T04:02:14.317ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
20482024-11-02T04:02:14.317ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
20492024-11-02T04:02:14.317ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
20502024-11-02T04:02:14.317ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte0" rules = []
20512024-11-02T04:02:14.317ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
20522024-11-02T04:02:14.317ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
20532024-11-02T04:02:14.317ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 381 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:43188 req_id = 3fad9943-7aad-47c1-9b07-bd9b4ee718e8 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
20542024-11-02T04:02:30.039ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:50477
20552024-11-02T04:02:30.039ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 30 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:50477 req_id = 16e8b92f-08f5-4e6d-b80f-c3ee30f5ac97 response_code = 200 uri = /baseboard
20562024-11-02T04:02:42.847ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:55331
20572024-11-02T04:02:42.847ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 64 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:55331 req_id = 6927824c-a71a-4dec-b2d8-a801811103f4 response_code = 200 uri = /vpc-routes
20582024-11-02T04:02:42.862ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
20592024-11-02T04:02:42.862ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 56 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:55331 req_id = eae48dbd-713e-458d-acc7-377dd24affeb response_code = 204 uri = /eip-gateways
20602024-11-02T04:02:42.871ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 162 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:55331 req_id = 3fc0090a-9f3b-448b-829a-505d8d2f2113 response_code = 204 uri = /vpc-routes
20612024-11-02T04:02:42.915ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:45968
20622024-11-02T04:02:42.916ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 115 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:45968 req_id = cc35cce2-4e7d-43ed-b1ab-033acea8b840 response_code = 200 uri = /v2p
20632024-11-02T04:02:42.963ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:61898
20642024-11-02T04:02:42.963ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
20652024-11-02T04:02:43.002ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
20662024-11-02T04:02:43.057ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 93465 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:61898 req_id = 985b51a6-5a49-461a-9094-391abf53047c response_code = 204 uri = /switch-ports
20672024-11-02T04:02:43.057ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 123 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:61898 req_id = 131ecaf3-4ef2-41e9-b3e4-7946a9b41354 response_code = 200 uri = /network-bootstore-config
20682024-11-02T04:02:43.626ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:49688
20692024-11-02T04:02:43.626ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 71 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:49688 req_id = 49fccb51-25b8-43c0-ae4d-9533ba53e10e response_code = 200 uri = /vpc-routes
20702024-11-02T04:02:43.669ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
20712024-11-02T04:02:43.669ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 88 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:49688 req_id = 4bc8b3d3-a233-45d5-bf40-ea9830c017fd response_code = 204 uri = /eip-gateways
20722024-11-02T04:02:43.678ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 264 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:49688 req_id = fad8accb-2eba-47b3-9e74-be3b451afdb8 response_code = 204 uri = /vpc-routes
20732024-11-02T04:02:43.707ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:63271
20742024-11-02T04:02:43.707ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
20752024-11-02T04:02:43.752ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
20762024-11-02T04:02:43.806ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 98289 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:63271 req_id = 72446606-47f4-4052-933e-a54ae21aeafc response_code = 204 uri = /switch-ports
20772024-11-02T04:02:43.806ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:59128
20782024-11-02T04:02:43.806ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:44508
20792024-11-02T04:02:43.806ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 240 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:63271 req_id = 7cf88771-22a2-4e41-883f-ad1ba4b8f094 response_code = 200 uri = /network-bootstore-config
20802024-11-02T04:02:43.806ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 101 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:44508 req_id = ea136d62-7070-4f72-a1aa-f710feb829b7 response_code = 200 uri = /vpc-routes
20812024-11-02T04:02:43.806ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 185 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:59128 req_id = 2449784b-190b-4aed-a4af-92660f93e362 response_code = 200 uri = /v2p
20822024-11-02T04:02:43.815ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:59931
20832024-11-02T04:02:43.815ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 156 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:59931 req_id = db8205c8-be73-4017-a420-dfec54141fe5 response_code = 200 uri = /v2p
20842024-11-02T04:02:43.822ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
20852024-11-02T04:02:43.823ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 57 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:44508 req_id = 7cba6d57-9f30-48b8-a46b-a4db6e2620c4 response_code = 204 uri = /eip-gateways
20862024-11-02T04:02:43.831ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 145 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:44508 req_id = 7bc30acd-61cc-4afd-9d80-029275e52231 response_code = 204 uri = /vpc-routes
20872024-11-02T04:02:43.838ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:39991
20882024-11-02T04:02:43.838ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
20892024-11-02T04:02:43.875ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
20902024-11-02T04:02:43.935ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 96443 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:39991 req_id = 1a8824f5-9ed5-4c67-8818-64a9f2b5b69a response_code = 204 uri = /switch-ports
20912024-11-02T04:02:43.935ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 115 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:39991 req_id = 8ffd6087-2882-4b3b-b86f-30fa76f80cc2 response_code = 200 uri = /network-bootstore-config
20922024-11-02T04:03:00.073ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:53583
20932024-11-02T04:03:00.073ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 42 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:53583 req_id = 11cd3646-1806-4d04-91b5-0b1d53d03bf5 response_code = 200 uri = /baseboard
20942024-11-02T04:03:12.956ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:47154
20952024-11-02T04:03:12.956ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 76 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:47154 req_id = f277c19c-c69f-4956-8eff-dba18007b84f response_code = 200 uri = /vpc-routes
20962024-11-02T04:03:13.003ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
20972024-11-02T04:03:13.003ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 82 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:47154 req_id = e41ce71f-0a28-4383-929a-84bc4c65a35a response_code = 204 uri = /eip-gateways
20982024-11-02T04:03:13.012ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 255 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:47154 req_id = 773cc292-5897-469a-b9cf-3c484bcc72a3 response_code = 204 uri = /vpc-routes
20992024-11-02T04:03:13.088ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:37638
21002024-11-02T04:03:13.088ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 142 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:37638 req_id = e0db73c0-b854-4b20-afe7-92dcd1e364e1 response_code = 200 uri = /v2p
21012024-11-02T04:03:13.140ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:58912
21022024-11-02T04:03:13.140ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
21032024-11-02T04:03:13.180ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
21042024-11-02T04:03:13.238ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 98311 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:58912 req_id = afe301b9-b106-49ff-8c16-4a32be00cdb6 response_code = 204 uri = /switch-ports
21052024-11-02T04:03:13.238ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 97 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:58912 req_id = 6125ab05-5dc6-4adc-bd06-bf278897ed4a response_code = 200 uri = /network-bootstore-config
21062024-11-02T04:03:13.753ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:56044
21072024-11-02T04:03:13.753ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 94 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:56044 req_id = 2100064c-edd8-439f-8785-7876cb98bc3a response_code = 200 uri = /vpc-routes
21082024-11-02T04:03:13.779ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
21092024-11-02T04:03:13.779ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 97 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:56044 req_id = d77fa9d2-05a1-40d9-abcc-6602beff06aa response_code = 204 uri = /eip-gateways
21102024-11-02T04:03:13.788ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 200 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:56044 req_id = f2b5dbd5-4ac3-4ae4-84d8-4834eecd9858 response_code = 204 uri = /vpc-routes
21112024-11-02T04:03:13.873ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:57153
21122024-11-02T04:03:13.873ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 53 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:57153 req_id = ef0f0da9-09a7-42d9-b348-40b51a5bf2f1 response_code = 200 uri = /vpc-routes
21132024-11-02T04:03:13.915ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
21142024-11-02T04:03:13.915ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 61 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:57153 req_id = cbe61aff-b754-4490-9a0e-c0b8fb3fb059 response_code = 204 uri = /eip-gateways
21152024-11-02T04:03:13.923ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:52373
21162024-11-02T04:03:13.923ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 145 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:52373 req_id = 7673a7b1-e770-45a2-ba0c-8af82f71921d response_code = 200 uri = /v2p
21172024-11-02T04:03:13.924ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 132 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:57153 req_id = d7344fc1-950f-47a0-97f4-289cee5b354b response_code = 204 uri = /vpc-routes
21182024-11-02T04:03:13.942ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:38617
21192024-11-02T04:03:13.942ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
21202024-11-02T04:03:13.978ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
21212024-11-02T04:03:14.037ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 94500 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:38617 req_id = 05e92edf-055b-48fb-80a3-38b09749b18d response_code = 204 uri = /switch-ports
21222024-11-02T04:03:14.037ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:36205
21232024-11-02T04:03:14.037ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 118 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:36205 req_id = 4c3c4087-8af6-40f7-9e2c-3c05e003a6e9 response_code = 200 uri = /v2p
21242024-11-02T04:03:14.065ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 157 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:38617 req_id = a3e0bc0e-9864-4a3e-b41e-98fe96f5d9d7 response_code = 200 uri = /network-bootstore-config
21252024-11-02T04:03:14.135ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:60374
21262024-11-02T04:03:14.135ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
21272024-11-02T04:03:14.173ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
21282024-11-02T04:03:14.224ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 88965 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:60374 req_id = 4a78ed93-4279-4082-b8c0-a0315b3e38f8 response_code = 204 uri = /switch-ports
21292024-11-02T04:03:14.224ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 109 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:60374 req_id = 538427eb-5686-4df2-9366-f60f0790eb26 response_code = 200 uri = /network-bootstore-config
21302024-11-02T04:03:30.112ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:39667
21312024-11-02T04:03:30.112ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 94 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:39667 req_id = 8722afd9-ab28-4430-84d5-4bbe7cf4532c response_code = 200 uri = /baseboard
21322024-11-02T04:03:42.846ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:59402
21332024-11-02T04:03:42.847ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 67 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:59402 req_id = c1d69b6b-9673-44ee-92d0-4f015fc33827 response_code = 200 uri = /vpc-routes
21342024-11-02T04:03:42.893ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
21352024-11-02T04:03:42.893ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 78 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:59402 req_id = 8b529162-c01e-4276-afe6-390b8a03608b response_code = 204 uri = /eip-gateways
21362024-11-02T04:03:42.901ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 173 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:59402 req_id = 4669b38c-0dda-46e5-8f28-54e99aedce16 response_code = 204 uri = /vpc-routes
21372024-11-02T04:03:42.995ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:52496
21382024-11-02T04:03:42.996ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 133 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:52496 req_id = e4a91c60-eb71-4581-bc5c-bcd6b67f7a13 response_code = 200 uri = /v2p
21392024-11-02T04:03:43.004ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:54072
21402024-11-02T04:03:43.004ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
21412024-11-02T04:03:43.043ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
21422024-11-02T04:03:43.101ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 96659 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:54072 req_id = bef1e1fd-d37a-437f-97fa-d55a740ea1cc response_code = 204 uri = /switch-ports
21432024-11-02T04:03:43.101ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 117 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:54072 req_id = 76eda4fc-e919-4b30-b928-d4665cf8dfb8 response_code = 200 uri = /network-bootstore-config
21442024-11-02T04:03:43.613ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:65508
21452024-11-02T04:03:43.613ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 82 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:65508 req_id = 7c388c9c-1f93-4ac4-aab2-82e4566382e5 response_code = 200 uri = /vpc-routes
21462024-11-02T04:03:43.658ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
21472024-11-02T04:03:43.658ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 109 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:65508 req_id = e5732ac6-c08c-46c9-8d04-94f9d4c12c27 response_code = 204 uri = /eip-gateways
21482024-11-02T04:03:43.667ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 186 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:65508 req_id = e960b413-4f96-4106-af4a-e70883b949f4 response_code = 204 uri = /vpc-routes
21492024-11-02T04:03:43.693ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:36045
21502024-11-02T04:03:43.693ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 149 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:36045 req_id = f6ade837-f642-445d-9039-cf6c2f16743a response_code = 200 uri = /v2p
21512024-11-02T04:03:43.711ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:48228
21522024-11-02T04:03:43.711ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 43 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:48228 req_id = ef50665b-4d78-43b2-8f6c-7611bf650604 response_code = 200 uri = /vpc-routes
21532024-11-02T04:03:43.724ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:63578
21542024-11-02T04:03:43.724ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
21552024-11-02T04:03:43.764ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
21562024-11-02T04:03:43.823ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 99272 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:63578 req_id = 948ec3b6-f18c-4cd6-9314-2c1cc5d08725 response_code = 204 uri = /switch-ports
21572024-11-02T04:03:43.823ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:54833
21582024-11-02T04:03:43.823ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
21592024-11-02T04:03:43.823ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 167 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:48228 req_id = 45b029db-646d-42c5-99cb-91bde8627f58 response_code = 204 uri = /eip-gateways
21602024-11-02T04:03:43.824ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 223 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:63578 req_id = a13b5c22-3b3a-4b1b-913c-994149e7029e response_code = 200 uri = /network-bootstore-config
21612024-11-02T04:03:43.824ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 230 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:54833 req_id = d9601459-ea95-4635-b52b-bb21c9741fc5 response_code = 200 uri = /v2p
21622024-11-02T04:03:43.827ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:52945
21632024-11-02T04:03:43.827ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
21642024-11-02T04:03:43.865ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
21652024-11-02T04:03:43.928ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 100816 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:52945 req_id = 274d52f9-7587-41e9-8d03-93a6f74456af response_code = 204 uri = /switch-ports
21662024-11-02T04:03:43.928ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 344 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:48228 req_id = 4093a380-4fec-4d39-8fb4-d493600f1f3d response_code = 204 uri = /vpc-routes
21672024-11-02T04:03:43.928ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 257 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:52945 req_id = 51b44d3c-7ad5-4423-a9f5-a468e0b0020c response_code = 200 uri = /network-bootstore-config
21682024-11-02T04:04:00.146ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:33975
21692024-11-02T04:04:00.146ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 31 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:33975 req_id = 582407ec-dc94-4a48-b77f-fa498ae5ee4d response_code = 200 uri = /baseboard
21702024-11-02T04:04:12.960ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:64846
21712024-11-02T04:04:12.960ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 71 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:64846 req_id = 037433f4-6862-4441-b185-61b608e04384 response_code = 200 uri = /vpc-routes
21722024-11-02T04:04:13.005ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
21732024-11-02T04:04:13.005ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 83 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:64846 req_id = 5de7918e-2bd9-4ab2-aaed-bf61fd481713 response_code = 204 uri = /eip-gateways
21742024-11-02T04:04:13.037ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:39173
21752024-11-02T04:04:13.037ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 136 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:39173 req_id = 5fe4a51e-ccbf-4e59-a654-a0db35a53fd2 response_code = 200 uri = /v2p
21762024-11-02T04:04:13.046ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 142 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:64846 req_id = 15b75446-53a0-41cf-808c-09102366e9da response_code = 204 uri = /vpc-routes
21772024-11-02T04:04:13.203ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:41124
21782024-11-02T04:04:13.203ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
21792024-11-02T04:04:13.244ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
21802024-11-02T04:04:13.303ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 100453 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:41124 req_id = 6c3e9af2-473a-44c2-8c68-db260b044eaa response_code = 204 uri = /switch-ports
21812024-11-02T04:04:13.304ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 109 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:41124 req_id = 9c24c5c7-d714-4b27-9ff9-e642f7137933 response_code = 200 uri = /network-bootstore-config
21822024-11-02T04:04:13.682ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:60851
21832024-11-02T04:04:13.682ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 88 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:60851 req_id = f958dfe9-fbf0-4f1e-8c22-c1c9f84b52c8 response_code = 200 uri = /vpc-routes
21842024-11-02T04:04:13.734ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
21852024-11-02T04:04:13.734ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 102 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:60851 req_id = e2f1803c-b32e-4cbe-be9c-24794e4865cc response_code = 204 uri = /eip-gateways
21862024-11-02T04:04:13.824ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:58183
21872024-11-02T04:04:13.824ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 144 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:58183 req_id = 3bedeccc-dc39-43a6-8a57-b1e5e0446f8e response_code = 200 uri = /v2p
21882024-11-02T04:04:13.830ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 212 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:60851 req_id = 6e41070b-f401-46dc-9996-a1d3abec7dea response_code = 204 uri = /vpc-routes
21892024-11-02T04:04:13.841ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:53745
21902024-11-02T04:04:13.841ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 31 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:53745 req_id = e7802186-9fe8-428e-aa85-1b70aa0b9eea response_code = 200 uri = /vpc-routes
21912024-11-02T04:04:13.899ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
21922024-11-02T04:04:13.899ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 144 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:53745 req_id = 31045fd7-be1f-444d-a9b0-c45c0233e468 response_code = 204 uri = /eip-gateways
21932024-11-02T04:04:13.909ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 230 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:53745 req_id = 6dffc9b5-7533-4676-b35e-a57d64436f6e response_code = 204 uri = /vpc-routes
21942024-11-02T04:04:13.909ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:64818
21952024-11-02T04:04:13.909ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
21962024-11-02T04:04:13.959ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
21972024-11-02T04:04:14.033ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 123882 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:64818 req_id = f91b3db5-05c4-47fc-ab97-87d68d924bf6 response_code = 204 uri = /switch-ports
21982024-11-02T04:04:14.033ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:56998
21992024-11-02T04:04:14.033ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:34121
22002024-11-02T04:04:14.033ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
22012024-11-02T04:04:14.033ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 239 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:64818 req_id = 887b3308-dc28-45a3-bc89-de629d05eeb3 response_code = 200 uri = /network-bootstore-config
22022024-11-02T04:04:14.039ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 5863 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:34121 req_id = 5284113c-e5d6-4f8f-950f-c577ecf07abf response_code = 200 uri = /v2p
22032024-11-02T04:04:14.076ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
22042024-11-02T04:04:14.137ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 103972 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:56998 req_id = 63a119c0-ee0b-4347-b6f8-bd5d0a8fa800 response_code = 204 uri = /switch-ports
22052024-11-02T04:04:14.137ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 137 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:56998 req_id = 124983cc-9b29-4238-9483-7d9a1d7248ed response_code = 200 uri = /network-bootstore-config
22062024-11-02T04:04:30.179ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:43811
22072024-11-02T04:04:30.179ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 39 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:43811 req_id = 05b798c5-f837-4c5d-a8e1-924fbdb82d95 response_code = 200 uri = /baseboard
22082024-11-02T04:04:37.316ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 498 n_new_samples = 6 n_overflow_samples = 4 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22092024-11-02T04:04:37.320ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 498 n_new_samples = 6 n_overflow_samples = 4 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22102024-11-02T04:04:37.321ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 498 n_new_samples = 6 n_overflow_samples = 4 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22112024-11-02T04:04:37.322ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 498 n_new_samples = 6 n_overflow_samples = 4 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22122024-11-02T04:04:37.322ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 498 n_new_samples = 6 n_overflow_samples = 4 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22132024-11-02T04:04:37.328ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 498 n_new_samples = 6 n_overflow_samples = 4 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22142024-11-02T04:04:37.328ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 498 n_new_samples = 6 n_overflow_samples = 4 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22152024-11-02T04:04:37.371ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 498 n_new_samples = 6 n_overflow_samples = 4 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22162024-11-02T04:04:37.372ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 498 n_new_samples = 6 n_overflow_samples = 4 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22172024-11-02T04:04:42.830ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:56399
22182024-11-02T04:04:42.830ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 78 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:56399 req_id = 92b23017-0beb-4340-b28f-e783ef6700bf response_code = 200 uri = /vpc-routes
22192024-11-02T04:04:42.872ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
22202024-11-02T04:04:42.872ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 87 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:56399 req_id = e0c6ba93-cce8-4735-a68f-4443026cdda5 response_code = 204 uri = /eip-gateways
22212024-11-02T04:04:42.881ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 171 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:56399 req_id = fecc71ff-2e91-4353-8ea3-3c039c5d699e response_code = 204 uri = /vpc-routes
22222024-11-02T04:04:43.017ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:46943
22232024-11-02T04:04:43.017ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 141 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:46943 req_id = 43cb34b3-20d2-4803-bf06-567d6ce46a4d response_code = 200 uri = /v2p
22242024-11-02T04:04:43.031ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:37088
22252024-11-02T04:04:43.031ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
22262024-11-02T04:04:43.067ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
22272024-11-02T04:04:43.126ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 94904 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:37088 req_id = 51e65725-0e64-40c1-b2c2-29d50e975017 response_code = 204 uri = /switch-ports
22282024-11-02T04:04:43.126ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 104 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:37088 req_id = 50f640dd-add3-4aa2-a790-5d84fd116126 response_code = 200 uri = /network-bootstore-config
22292024-11-02T04:04:43.664ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:60661
22302024-11-02T04:04:43.664ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 89 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:60661 req_id = f5fa9a74-04ca-4d18-9423-5d16e2c33c12 response_code = 200 uri = /vpc-routes
22312024-11-02T04:04:43.708ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:57775
22322024-11-02T04:04:43.708ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
22332024-11-02T04:04:43.754ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
22342024-11-02T04:04:43.818ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 110330 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:57775 req_id = 479bd061-9a39-4e7d-a82f-b358775b03d8 response_code = 204 uri = /switch-ports
22352024-11-02T04:04:43.818ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
22362024-11-02T04:04:43.818ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 110 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:60661 req_id = de83daba-0a4c-48d6-88d8-84ab8d7ab042 response_code = 204 uri = /eip-gateways
22372024-11-02T04:04:43.818ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:44881
22382024-11-02T04:04:43.819ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:52296
22392024-11-02T04:04:43.819ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 243 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:57775 req_id = aa8e608c-8f9f-441c-a12c-50fbc60fdc73 response_code = 200 uri = /network-bootstore-config
22402024-11-02T04:04:43.819ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 108 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:52296 req_id = 91d0d67b-e138-41dd-8df3-f593b576ace1 response_code = 200 uri = /vpc-routes
22412024-11-02T04:04:43.819ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 129 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:44881 req_id = 79942b11-4902-4f61-a9ed-a9b75c150fcd response_code = 200 uri = /v2p
22422024-11-02T04:04:43.829ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 217 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:60661 req_id = 5cf97e3a-9e49-4749-9fe7-5bf52058c2a6 response_code = 204 uri = /vpc-routes
22432024-11-02T04:04:43.833ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
22442024-11-02T04:04:43.833ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 41 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:52296 req_id = 327ffc7a-e0b9-4908-9974-a2f04e69223e response_code = 204 uri = /eip-gateways
22452024-11-02T04:04:43.892ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:53568
22462024-11-02T04:04:43.893ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 127 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:53568 req_id = 6d1c1272-43c7-4697-bc6b-1e06b240f5e7 response_code = 200 uri = /v2p
22472024-11-02T04:04:43.895ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 131 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:52296 req_id = d9093dd2-eefe-44f6-ac40-1b9858dcfaae response_code = 204 uri = /vpc-routes
22482024-11-02T04:04:43.979ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:41980
22492024-11-02T04:04:43.979ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
22502024-11-02T04:04:44.020ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
22512024-11-02T04:04:44.082ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 103199 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:41980 req_id = 6402b312-6f8c-4480-8530-0a5d5ea2f7c4 response_code = 204 uri = /switch-ports
22522024-11-02T04:04:44.082ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 109 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:41980 req_id = ecebded7-0673-4996-b095-321d553d01b9 response_code = 200 uri = /network-bootstore-config
22532024-11-02T04:04:45.350ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22542024-11-02T04:04:45.350ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22552024-11-02T04:04:45.351ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22562024-11-02T04:04:47.244ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22572024-11-02T04:04:47.248ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22582024-11-02T04:04:47.257ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22592024-11-02T04:04:47.281ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22602024-11-02T04:04:47.313ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22612024-11-02T04:04:47.317ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22622024-11-02T04:04:47.318ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22632024-11-02T04:04:47.322ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22642024-11-02T04:04:47.323ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22652024-11-02T04:04:47.324ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22662024-11-02T04:04:47.324ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22672024-11-02T04:04:47.330ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22682024-11-02T04:04:47.331ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22692024-11-02T04:04:47.373ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22702024-11-02T04:04:47.374ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22712024-11-02T04:04:55.354ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22722024-11-02T04:04:55.355ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22732024-11-02T04:04:55.356ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22742024-11-02T04:04:57.249ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22752024-11-02T04:04:57.250ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22762024-11-02T04:04:57.259ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22772024-11-02T04:04:57.283ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22782024-11-02T04:04:57.315ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22792024-11-02T04:04:57.319ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22802024-11-02T04:04:57.319ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22812024-11-02T04:04:57.323ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22822024-11-02T04:04:57.325ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22832024-11-02T04:04:57.326ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22842024-11-02T04:04:57.327ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22852024-11-02T04:04:57.332ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22862024-11-02T04:04:57.333ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22872024-11-02T04:04:57.375ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22882024-11-02T04:04:57.376ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22892024-11-02T04:05:00.217ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:54822
22902024-11-02T04:05:00.217ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 47 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:54822 req_id = f427f4c9-7d26-488b-a015-6b6c441675c9 response_code = 200 uri = /baseboard
22912024-11-02T04:05:05.359ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22922024-11-02T04:05:05.359ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22932024-11-02T04:05:05.360ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22942024-11-02T04:05:07.253ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22952024-11-02T04:05:07.254ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22962024-11-02T04:05:07.261ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22972024-11-02T04:05:07.285ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22982024-11-02T04:05:07.317ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
22992024-11-02T04:05:07.321ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23002024-11-02T04:05:07.322ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23012024-11-02T04:05:07.325ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23022024-11-02T04:05:07.326ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23032024-11-02T04:05:07.328ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23042024-11-02T04:05:07.329ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23052024-11-02T04:05:07.333ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23062024-11-02T04:05:07.335ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23072024-11-02T04:05:07.376ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23082024-11-02T04:05:07.378ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23092024-11-02T04:05:09.307ZINFOSledAgent (auto-cleanup-task): running automatic periodic zone bundle cleanup file = sled-agent/src/zone_bundle.rs:197
23102024-11-02T04:05:09.307ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
23112024-11-02T04:05:09.340ZINFOSledAgent (auto-cleanup-task): top of bundle cleanup loop file = sled-agent/src/zone_bundle.rs:186 next_cleanup = Instant { tv_sec: 10413, tv_nsec: 317270664 } time_to_next_cleanup = 599.99999624s
23122024-11-02T04:05:13.003ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:63701
23132024-11-02T04:05:13.003ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 68 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:63701 req_id = f01852f2-bcee-42a1-9447-25d4be40e0b0 response_code = 200 uri = /vpc-routes
23142024-11-02T04:05:13.075ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:58281
23152024-11-02T04:05:13.075ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 118 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:58281 req_id = 725b8996-3d8a-45a6-93cd-be117d9fee4d response_code = 200 uri = /v2p
23162024-11-02T04:05:13.122ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
23172024-11-02T04:05:13.122ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 63 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:63701 req_id = 568dfca6-38f7-449f-b6e0-c6de16a0fb07 response_code = 204 uri = /eip-gateways
23182024-11-02T04:05:13.130ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 172 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:63701 req_id = 75e7e374-4fc3-4559-9650-3c4afe3b15ba response_code = 204 uri = /vpc-routes
23192024-11-02T04:05:13.235ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:59695
23202024-11-02T04:05:13.235ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
23212024-11-02T04:05:13.273ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
23222024-11-02T04:05:13.332ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 96631 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:59695 req_id = 3068aad5-041d-4753-8349-a6229e372dee response_code = 204 uri = /switch-ports
23232024-11-02T04:05:13.332ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 105 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:59695 req_id = 6ed486ed-b496-4b9c-933a-92658b4086f7 response_code = 200 uri = /network-bootstore-config
23242024-11-02T04:05:13.772ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:38093
23252024-11-02T04:05:13.773ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 87 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:38093 req_id = 2379ce9f-93a4-4f1a-9b7c-20834932d79e response_code = 200 uri = /vpc-routes
23262024-11-02T04:05:13.824ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
23272024-11-02T04:05:13.824ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 81 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:38093 req_id = 48af9db6-fbb7-499d-9adc-f3984cc5984b response_code = 204 uri = /eip-gateways
23282024-11-02T04:05:13.832ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:41826
23292024-11-02T04:05:13.832ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 38 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:41826 req_id = ed512c37-0db9-4aef-88e1-1bed085ab0d7 response_code = 200 uri = /vpc-routes
23302024-11-02T04:05:13.833ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 201 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:38093 req_id = bf61f8c9-c0c1-49f3-b35a-b0c7870b3e5f response_code = 204 uri = /vpc-routes
23312024-11-02T04:05:13.902ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
23322024-11-02T04:05:13.902ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 69 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:41826 req_id = 08a9a984-4aad-4d4d-af23-e3f55ebbbf03 response_code = 204 uri = /eip-gateways
23332024-11-02T04:05:13.918ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 182 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:41826 req_id = 8eb8105e-d94b-4d4e-b327-9f0baa7e8a53 response_code = 204 uri = /vpc-routes
23342024-11-02T04:05:14.004ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:34074
23352024-11-02T04:05:14.004ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
23362024-11-02T04:05:14.047ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
23372024-11-02T04:05:14.112ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 108087 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:34074 req_id = 6caa684d-1ff9-4a2e-bef7-4cd876fa04c1 response_code = 204 uri = /switch-ports
23382024-11-02T04:05:14.112ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:51516
23392024-11-02T04:05:14.112ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:61506
23402024-11-02T04:05:14.112ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 199 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:61506 req_id = 22fa6ea1-0213-4619-8f11-f63e51b8fd19 response_code = 200 uri = /v2p
23412024-11-02T04:05:14.113ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 223 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:34074 req_id = 67114674-f67e-4b79-abe0-fab00390a57d response_code = 200 uri = /network-bootstore-config
23422024-11-02T04:05:14.113ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 216 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:51516 req_id = ebcf4c1e-9bcc-4a7a-99d6-f6b8a4742d98 response_code = 200 uri = /v2p
23432024-11-02T04:05:14.128ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:57790
23442024-11-02T04:05:14.128ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
23452024-11-02T04:05:14.169ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
23462024-11-02T04:05:14.228ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 99494 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:57790 req_id = ad6c3b03-1423-4aa1-838e-afcd566469f1 response_code = 204 uri = /switch-ports
23472024-11-02T04:05:14.228ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 209 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:57790 req_id = 08b8fdd7-a06e-4408-ae31-44a27df70f16 response_code = 200 uri = /network-bootstore-config
23482024-11-02T04:05:15.362ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23492024-11-02T04:05:15.363ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23502024-11-02T04:05:15.364ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23512024-11-02T04:05:17.257ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23522024-11-02T04:05:17.258ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23532024-11-02T04:05:17.263ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23542024-11-02T04:05:17.287ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23552024-11-02T04:05:17.319ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23562024-11-02T04:05:17.323ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23572024-11-02T04:05:17.324ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23582024-11-02T04:05:17.326ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23592024-11-02T04:05:17.328ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23602024-11-02T04:05:17.330ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23612024-11-02T04:05:17.331ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23622024-11-02T04:05:17.336ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23632024-11-02T04:05:17.336ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23642024-11-02T04:05:17.378ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23652024-11-02T04:05:17.379ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23662024-11-02T04:05:23.854ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
23672024-11-02T04:05:25.367ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23682024-11-02T04:05:25.368ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23692024-11-02T04:05:25.368ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23702024-11-02T04:05:27.261ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23712024-11-02T04:05:27.262ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23722024-11-02T04:05:27.265ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23732024-11-02T04:05:27.290ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23742024-11-02T04:05:27.320ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23752024-11-02T04:05:27.325ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23762024-11-02T04:05:27.326ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23772024-11-02T04:05:27.328ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23782024-11-02T04:05:27.331ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23792024-11-02T04:05:27.332ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23802024-11-02T04:05:27.333ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23812024-11-02T04:05:27.338ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23822024-11-02T04:05:27.338ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23832024-11-02T04:05:27.379ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23842024-11-02T04:05:27.381ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23852024-11-02T04:05:30.249ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:53942
23862024-11-02T04:05:30.249ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 108 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:53942 req_id = 5e156c63-e469-4ccd-9340-ccc7a50dbd06 response_code = 200 uri = /baseboard
23872024-11-02T04:05:35.371ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23882024-11-02T04:05:35.371ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23892024-11-02T04:05:35.372ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23902024-11-02T04:05:37.266ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23912024-11-02T04:05:37.266ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23922024-11-02T04:05:37.267ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23932024-11-02T04:05:37.292ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23942024-11-02T04:05:37.322ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23952024-11-02T04:05:37.328ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23962024-11-02T04:05:37.328ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23972024-11-02T04:05:37.329ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23982024-11-02T04:05:37.333ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
23992024-11-02T04:05:37.334ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24002024-11-02T04:05:37.335ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24012024-11-02T04:05:37.339ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24022024-11-02T04:05:37.341ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24032024-11-02T04:05:37.381ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24042024-11-02T04:05:37.382ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24052024-11-02T04:05:42.791ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:63267
24062024-11-02T04:05:42.791ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 87 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:63267 req_id = 9c46670b-b6fa-4df3-9cd8-b0fc7c25b6f0 response_code = 200 uri = /vpc-routes
24072024-11-02T04:05:42.806ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
24082024-11-02T04:05:42.806ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 69 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:63267 req_id = abe2e9e0-5ef3-4ba1-9837-3c3f03734059 response_code = 204 uri = /eip-gateways
24092024-11-02T04:05:42.814ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 176 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:63267 req_id = 8afde528-2ffe-45fc-b5a9-d2eb8dfac38b response_code = 204 uri = /vpc-routes
24102024-11-02T04:05:42.905ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:58626
24112024-11-02T04:05:42.905ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 174 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:58626 req_id = e0c7a9cc-5087-414d-824b-5cd3d33b13b3 response_code = 200 uri = /v2p
24122024-11-02T04:05:42.928ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:65314
24132024-11-02T04:05:42.928ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
24142024-11-02T04:05:42.968ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
24152024-11-02T04:05:43.028ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 100547 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:65314 req_id = 0c665f2c-6c53-4faf-b844-50674aaa149c response_code = 204 uri = /switch-ports
24162024-11-02T04:05:43.029ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 109 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:65314 req_id = 6463dd35-0c8d-47e3-95f2-5a6bf3297e41 response_code = 200 uri = /network-bootstore-config
24172024-11-02T04:05:43.631ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:40152
24182024-11-02T04:05:43.631ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 95 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:40152 req_id = 7550cd7a-df52-4414-9b7c-31ed56593d97 response_code = 200 uri = /vpc-routes
24192024-11-02T04:05:43.688ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
24202024-11-02T04:05:43.688ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 96 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:40152 req_id = f2b27c82-0b4b-4308-bbc2-32868111fcc0 response_code = 204 uri = /eip-gateways
24212024-11-02T04:05:43.699ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 230 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:40152 req_id = 3922c116-3870-418b-b216-2282caf82c70 response_code = 204 uri = /vpc-routes
24222024-11-02T04:05:43.767ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:40075
24232024-11-02T04:05:43.767ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 170 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:40075 req_id = 57d08947-5b57-4bb6-a4e2-9ea6a786ffde response_code = 200 uri = /v2p
24242024-11-02T04:05:43.788ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:64256
24252024-11-02T04:05:43.788ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 48 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:64256 req_id = 0d6d75aa-2f3f-49cb-9219-770e713bd4ca response_code = 200 uri = /vpc-routes
24262024-11-02T04:05:43.832ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:59275
24272024-11-02T04:05:43.832ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
24282024-11-02T04:05:43.888ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
24292024-11-02T04:05:43.969ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 137076 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:59275 req_id = 2cad8959-029d-407b-9774-a135f17ea3d6 response_code = 204 uri = /switch-ports
24302024-11-02T04:05:43.970ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:35919
24312024-11-02T04:05:43.970ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
24322024-11-02T04:05:43.970ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:33080
24332024-11-02T04:05:43.970ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 197 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:64256 req_id = 27fd87d3-b1a7-4850-aff2-f914d19e2a09 response_code = 204 uri = /eip-gateways
24342024-11-02T04:05:43.970ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 240 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:59275 req_id = b178673f-4f02-478c-836d-6f6a50e0263d response_code = 200 uri = /network-bootstore-config
24352024-11-02T04:05:43.970ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
24362024-11-02T04:05:43.970ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 200 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:33080 req_id = 83d1c93e-dc8f-48c0-aee9-5a8525cdc336 response_code = 200 uri = /v2p
24372024-11-02T04:05:43.981ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 540 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:64256 req_id = 0cee0ce2-3691-458e-a08e-b0eb090763f1 response_code = 204 uri = /vpc-routes
24382024-11-02T04:05:44.012ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
24392024-11-02T04:05:44.076ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 105886 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:35919 req_id = 94d44ff2-77e8-4fe4-8c75-1531e83db5d7 response_code = 204 uri = /switch-ports
24402024-11-02T04:05:44.076ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 167 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:35919 req_id = b49a7400-11f7-4381-bb94-f5189207ed31 response_code = 200 uri = /network-bootstore-config
24412024-11-02T04:05:45.374ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24422024-11-02T04:05:45.375ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24432024-11-02T04:05:45.376ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24442024-11-02T04:05:47.269ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24452024-11-02T04:05:47.270ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24462024-11-02T04:05:47.271ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24472024-11-02T04:05:47.295ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24482024-11-02T04:05:47.324ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24492024-11-02T04:05:47.330ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24502024-11-02T04:05:47.331ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24512024-11-02T04:05:47.331ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24522024-11-02T04:05:47.334ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24532024-11-02T04:05:47.337ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24542024-11-02T04:05:47.337ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24552024-11-02T04:05:47.340ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24562024-11-02T04:05:47.342ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24572024-11-02T04:05:47.384ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24582024-11-02T04:05:47.384ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24592024-11-02T04:05:55.379ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24602024-11-02T04:05:55.380ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24612024-11-02T04:05:55.381ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24622024-11-02T04:05:57.274ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24632024-11-02T04:05:57.282ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24642024-11-02T04:05:57.283ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24652024-11-02T04:05:57.297ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24662024-11-02T04:05:57.326ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24672024-11-02T04:05:57.333ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24682024-11-02T04:05:57.334ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24692024-11-02T04:05:57.334ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24702024-11-02T04:05:57.335ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24712024-11-02T04:05:57.339ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24722024-11-02T04:05:57.340ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24732024-11-02T04:05:57.343ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24742024-11-02T04:05:57.344ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24752024-11-02T04:05:57.385ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24762024-11-02T04:05:57.386ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24772024-11-02T04:06:00.287ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:54550
24782024-11-02T04:06:00.287ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 34 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:54550 req_id = 3e9a8ed0-3eaa-420e-b537-558b1fdde10c response_code = 200 uri = /baseboard
24792024-11-02T04:06:05.383ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24802024-11-02T04:06:05.383ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24812024-11-02T04:06:05.384ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24822024-11-02T04:06:07.279ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24832024-11-02T04:06:07.284ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24842024-11-02T04:06:07.284ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24852024-11-02T04:06:07.299ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24862024-11-02T04:06:07.328ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24872024-11-02T04:06:07.335ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24882024-11-02T04:06:07.335ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24892024-11-02T04:06:07.337ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24902024-11-02T04:06:07.337ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24912024-11-02T04:06:07.340ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24922024-11-02T04:06:07.342ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24932024-11-02T04:06:07.345ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24942024-11-02T04:06:07.345ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24952024-11-02T04:06:07.388ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24962024-11-02T04:06:07.389ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
24972024-11-02T04:06:12.982ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:55637
24982024-11-02T04:06:12.982ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 82 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:55637 req_id = 0080f1c0-e6c2-49fc-9fef-b6c265761909 response_code = 200 uri = /vpc-routes
24992024-11-02T04:06:13.138ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:59152
25002024-11-02T04:06:13.139ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 158 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:59152 req_id = 496a16a5-d534-46f4-b397-5a599889b7cd response_code = 200 uri = /v2p
25012024-11-02T04:06:13.179ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
25022024-11-02T04:06:13.179ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 65 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:55637 req_id = 2d62875d-41cb-47f8-9017-3e48cf0fe924 response_code = 204 uri = /eip-gateways
25032024-11-02T04:06:13.188ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 177 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:55637 req_id = a2014519-98d6-40b0-ae7e-b498529dc047 response_code = 204 uri = /vpc-routes
25042024-11-02T04:06:13.211ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:55969
25052024-11-02T04:06:13.211ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
25062024-11-02T04:06:13.249ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
25072024-11-02T04:06:13.303ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 92576 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:55969 req_id = e9315d28-1aaa-443d-ad2a-d23f98c34631 response_code = 204 uri = /switch-ports
25082024-11-02T04:06:13.304ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 101 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:55969 req_id = d7db8774-9dda-463e-a349-e97ac1e9fd36 response_code = 200 uri = /network-bootstore-config
25092024-11-02T04:06:13.806ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:59673
25102024-11-02T04:06:13.806ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 194 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:59673 req_id = de96ebd3-024e-497e-83cc-6b17659113bb response_code = 200 uri = /v2p
25112024-11-02T04:06:13.877ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:60885
25122024-11-02T04:06:13.877ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 50 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:60885 req_id = 682f7802-d5f7-4ab1-8000-f67029c8b806 response_code = 200 uri = /vpc-routes
25132024-11-02T04:06:13.924ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
25142024-11-02T04:06:13.924ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 95 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:60885 req_id = ee641068-efa2-4167-a1d8-fa8bb8e03f59 response_code = 204 uri = /eip-gateways
25152024-11-02T04:06:13.934ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 244 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:60885 req_id = 5c36177f-e249-4cfe-b760-9d10c1184daa response_code = 204 uri = /vpc-routes
25162024-11-02T04:06:13.945ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:47620
25172024-11-02T04:06:13.945ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 34 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:47620 req_id = 2dc43328-325a-48f5-bb9b-8db29192e03a response_code = 200 uri = /vpc-routes
25182024-11-02T04:06:13.993ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
25192024-11-02T04:06:13.993ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 129 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:47620 req_id = 6e0e9cf2-687a-4bb9-b0a3-24724590a2d9 response_code = 204 uri = /eip-gateways
25202024-11-02T04:06:13.993ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:42823
25212024-11-02T04:06:13.993ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
25222024-11-02T04:06:14.044ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
25232024-11-02T04:06:14.111ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 117225 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:42823 req_id = 11c2f44a-ade2-4460-88de-06440b3d9e14 response_code = 204 uri = /switch-ports
25242024-11-02T04:06:14.111ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:55582
25252024-11-02T04:06:14.111ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 409 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:47620 req_id = 6a4f872c-aa2f-4b5a-987d-13cfe1954b5b response_code = 204 uri = /vpc-routes
25262024-11-02T04:06:14.111ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 317 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:42823 req_id = 41b194f2-edb0-4479-a7f2-a60e101094c9 response_code = 200 uri = /network-bootstore-config
25272024-11-02T04:06:14.111ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 270 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:55582 req_id = 0cf90107-64b6-4052-a773-d632997ed1bd response_code = 200 uri = /v2p
25282024-11-02T04:06:14.129ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:57778
25292024-11-02T04:06:14.129ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
25302024-11-02T04:06:14.171ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
25312024-11-02T04:06:14.229ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 99748 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:57778 req_id = f36bf00e-dd36-4da9-a51d-566e4154e2f1 response_code = 204 uri = /switch-ports
25322024-11-02T04:06:14.229ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 107 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:57778 req_id = 7466b246-01db-4f99-9217-ab74c5de2bab response_code = 200 uri = /network-bootstore-config
25332024-11-02T04:06:15.387ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25342024-11-02T04:06:15.388ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25352024-11-02T04:06:15.389ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25362024-11-02T04:06:17.284ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25372024-11-02T04:06:17.285ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25382024-11-02T04:06:17.286ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25392024-11-02T04:06:17.301ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25402024-11-02T04:06:17.330ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25412024-11-02T04:06:17.337ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25422024-11-02T04:06:17.337ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25432024-11-02T04:06:17.338ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25442024-11-02T04:06:17.339ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25452024-11-02T04:06:17.341ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25462024-11-02T04:06:17.344ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25472024-11-02T04:06:17.347ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25482024-11-02T04:06:17.347ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25492024-11-02T04:06:17.390ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25502024-11-02T04:06:17.390ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25512024-11-02T04:06:25.392ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25522024-11-02T04:06:25.393ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25532024-11-02T04:06:25.393ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25542024-11-02T04:06:27.287ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25552024-11-02T04:06:27.288ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25562024-11-02T04:06:27.289ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25572024-11-02T04:06:27.303ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25582024-11-02T04:06:27.332ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25592024-11-02T04:06:27.339ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25602024-11-02T04:06:27.339ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25612024-11-02T04:06:27.340ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25622024-11-02T04:06:27.341ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25632024-11-02T04:06:27.343ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25642024-11-02T04:06:27.346ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25652024-11-02T04:06:27.348ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25662024-11-02T04:06:27.349ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25672024-11-02T04:06:27.392ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25682024-11-02T04:06:27.392ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25692024-11-02T04:06:30.324ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:35933
25702024-11-02T04:06:30.324ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 65 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:35933 req_id = 56ee3913-70da-4857-90a3-de130e84b6de response_code = 200 uri = /baseboard
25712024-11-02T04:06:35.396ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25722024-11-02T04:06:35.397ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25732024-11-02T04:06:35.398ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25742024-11-02T04:06:37.291ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25752024-11-02T04:06:37.292ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25762024-11-02T04:06:37.292ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25772024-11-02T04:06:37.304ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25782024-11-02T04:06:37.334ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25792024-11-02T04:06:37.341ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25802024-11-02T04:06:37.342ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25812024-11-02T04:06:37.342ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25822024-11-02T04:06:37.343ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25832024-11-02T04:06:37.344ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25842024-11-02T04:06:37.348ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25852024-11-02T04:06:37.350ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25862024-11-02T04:06:37.350ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25872024-11-02T04:06:37.393ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25882024-11-02T04:06:37.394ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
25892024-11-02T04:06:42.885ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:46521
25902024-11-02T04:06:42.885ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 61 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:46521 req_id = 5d6d207a-8dc8-4099-8e8d-45f7d2ce4ca6 response_code = 200 uri = /vpc-routes
25912024-11-02T04:06:42.902ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
25922024-11-02T04:06:42.902ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 80 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:46521 req_id = 04042812-8416-470c-bfcb-cc99ec7a5bf4 response_code = 204 uri = /eip-gateways
25932024-11-02T04:06:42.912ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 175 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:46521 req_id = 599d37d7-06f2-423c-a07b-a0d89e521cb6 response_code = 204 uri = /vpc-routes
25942024-11-02T04:06:43.004ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:48865
25952024-11-02T04:06:43.005ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 118 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:48865 req_id = a60b540a-057b-4c96-9902-24cdc3ee0b8b response_code = 200 uri = /v2p
25962024-11-02T04:06:43.036ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:65305
25972024-11-02T04:06:43.036ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
25982024-11-02T04:06:43.077ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
25992024-11-02T04:06:43.137ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 101445 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:65305 req_id = feab58f5-80d0-4f60-a47e-4ec9c58f452c response_code = 204 uri = /switch-ports
26002024-11-02T04:06:43.137ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 98 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:65305 req_id = 49cc87c4-c009-4227-be23-678f8180f9e5 response_code = 200 uri = /network-bootstore-config
26012024-11-02T04:06:43.556ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:55691
26022024-11-02T04:06:43.556ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 84 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:55691 req_id = b6c2a4da-4426-4e84-a236-1c31efaee381 response_code = 200 uri = /vpc-routes
26032024-11-02T04:06:43.586ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
26042024-11-02T04:06:43.586ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 104 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:55691 req_id = 17bf7abe-9839-4731-bf10-3b4e161b16f9 response_code = 204 uri = /eip-gateways
26052024-11-02T04:06:43.595ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 216 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:55691 req_id = d969dca1-13ac-451a-aa1c-ea33f4d95d0f response_code = 204 uri = /vpc-routes
26062024-11-02T04:06:43.738ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:33030
26072024-11-02T04:06:43.739ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 149 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:33030 req_id = e0a9b20d-b16c-44a8-8624-2bee16a65e30 response_code = 200 uri = /v2p
26082024-11-02T04:06:43.759ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:42975
26092024-11-02T04:06:43.759ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
26102024-11-02T04:06:43.802ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
26112024-11-02T04:06:43.864ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 104663 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:42975 req_id = 85d05b0c-db2b-4880-bd8f-0d60896200f7 response_code = 204 uri = /switch-ports
26122024-11-02T04:06:43.864ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:60246
26132024-11-02T04:06:43.864ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:39008
26142024-11-02T04:06:43.864ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 136 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:60246 req_id = 3a19df55-dea2-4461-af4f-5c2ba779122f response_code = 200 uri = /vpc-routes
26152024-11-02T04:06:43.864ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 258 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:39008 req_id = b471a8c6-fdc7-43d6-b0d3-58addca1b14e response_code = 200 uri = /v2p
26162024-11-02T04:06:43.865ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 249 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:42975 req_id = 78d6ae1b-9916-4084-872e-26effe3526e7 response_code = 200 uri = /network-bootstore-config
26172024-11-02T04:06:43.880ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
26182024-11-02T04:06:43.880ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 70 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:60246 req_id = a0349030-c690-4fd4-9a37-ccfd20174f63 response_code = 204 uri = /eip-gateways
26192024-11-02T04:06:43.888ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 198 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:60246 req_id = f1028016-36cd-4a26-afc4-9af2a3a18a7f response_code = 204 uri = /vpc-routes
26202024-11-02T04:06:43.900ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:62192
26212024-11-02T04:06:43.900ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
26222024-11-02T04:06:43.942ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
26232024-11-02T04:06:44.005ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 105689 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:62192 req_id = 51d29c03-87d5-41c6-8c1c-f2b5aa7020dc response_code = 204 uri = /switch-ports
26242024-11-02T04:06:44.006ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 106 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:62192 req_id = 42950c5e-fa5f-4d66-8666-509ff9fce1a0 response_code = 200 uri = /network-bootstore-config
26252024-11-02T04:06:45.400ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26262024-11-02T04:06:45.401ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26272024-11-02T04:06:45.401ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26282024-11-02T04:06:47.295ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26292024-11-02T04:06:47.296ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26302024-11-02T04:06:47.296ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26312024-11-02T04:06:47.306ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26322024-11-02T04:06:47.336ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26332024-11-02T04:06:47.343ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26342024-11-02T04:06:47.343ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26352024-11-02T04:06:47.344ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26362024-11-02T04:06:47.344ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26372024-11-02T04:06:47.345ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26382024-11-02T04:06:47.350ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26392024-11-02T04:06:47.352ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26402024-11-02T04:06:47.352ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26412024-11-02T04:06:47.395ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26422024-11-02T04:06:47.395ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26432024-11-02T04:06:55.405ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26442024-11-02T04:06:55.405ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26452024-11-02T04:06:55.406ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26462024-11-02T04:06:57.298ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26472024-11-02T04:06:57.299ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26482024-11-02T04:06:57.300ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26492024-11-02T04:06:57.308ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26502024-11-02T04:06:57.338ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26512024-11-02T04:06:57.345ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26522024-11-02T04:06:57.346ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26532024-11-02T04:06:57.346ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26542024-11-02T04:06:57.347ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26552024-11-02T04:06:57.347ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26562024-11-02T04:06:57.352ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26572024-11-02T04:06:57.354ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26582024-11-02T04:06:57.354ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26592024-11-02T04:06:57.397ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26602024-11-02T04:06:57.397ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26612024-11-02T04:07:00.357ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:38645
26622024-11-02T04:07:00.357ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 77 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:38645 req_id = 65cca644-e027-4ebc-a07b-50474493bed4 response_code = 200 uri = /baseboard
26632024-11-02T04:07:05.409ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26642024-11-02T04:07:05.409ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26652024-11-02T04:07:05.410ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26662024-11-02T04:07:07.302ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26672024-11-02T04:07:07.303ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26682024-11-02T04:07:07.304ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26692024-11-02T04:07:07.311ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26702024-11-02T04:07:07.340ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26712024-11-02T04:07:07.346ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26722024-11-02T04:07:07.347ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26732024-11-02T04:07:07.348ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26742024-11-02T04:07:07.348ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26752024-11-02T04:07:07.349ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26762024-11-02T04:07:07.354ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26772024-11-02T04:07:07.355ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26782024-11-02T04:07:07.356ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26792024-11-02T04:07:07.399ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26802024-11-02T04:07:07.399ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
26812024-11-02T04:07:12.901ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:46110
26822024-11-02T04:07:12.901ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 67 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:46110 req_id = 607d0fed-6876-4cd8-9850-a0e1ce50a40f response_code = 200 uri = /vpc-routes
26832024-11-02T04:07:12.953ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
26842024-11-02T04:07:12.953ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 81 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:46110 req_id = b1f731be-c034-432d-a83a-992716050667 response_code = 204 uri = /eip-gateways
26852024-11-02T04:07:13.036ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:38043
26862024-11-02T04:07:13.037ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 148 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:38043 req_id = f35e6798-f7db-4390-8c9e-703acb8da3db response_code = 200 uri = /v2p
26872024-11-02T04:07:13.045ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 148 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:46110 req_id = 2258f0dd-e693-4db7-8f8c-02f3c7f5fdf0 response_code = 204 uri = /vpc-routes
26882024-11-02T04:07:13.131ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:59756
26892024-11-02T04:07:13.132ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
26902024-11-02T04:07:13.174ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
26912024-11-02T04:07:13.237ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 105859 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:59756 req_id = aead5c68-eed9-4e0d-950e-a231c4f52763 response_code = 204 uri = /switch-ports
26922024-11-02T04:07:13.238ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:36945
26932024-11-02T04:07:13.238ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:41290
26942024-11-02T04:07:13.238ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
26952024-11-02T04:07:13.238ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
26962024-11-02T04:07:13.238ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/6308f1ec-c08e-4282-a1c2-db37d6b8a044/config/omicron-zones.json file = common/src/ledger.rs:177
26972024-11-02T04:07:13.238ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 273 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:59756 req_id = 166e97b3-dd69-4a17-b662-a7fed4ffbbbb response_code = 200 uri = /network-bootstore-config
26982024-11-02T04:07:13.238ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:731 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 27d7c9ec-b8a8-4b31-ae12-c49ad36788f1, kind: Service { id: 7f79c18f-7b53-4b8a-86fc-bb23b923ea1d }, name: Name("external-dns-7f79c18f-7b53-4b8a-86fc-bb23b923ea1d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 238, 20])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 29b92946-7588-4b89-8147-322049e1f231, kind: Service { id: f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b }, name: Name("external-dns-f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 239, 77])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0cd6826d-84a0-4cd5-b7c6-32cb83627e6c, kind: Service { id: 9325209d-81b8-49e4-be6a-cedff3ef78c1 }, name: Name("nexus-9325209d-81b8-49e4-be6a-cedff3ef78c1"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 192, 104])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ec778b7a-3947-4776-98f2-be2ced817d66, kind: Service { id: d2bdfdcf-c904-463a-87e2-60b209fd743f }, name: Name("nexus-d2bdfdcf-c904-463a-87e2-60b209fd743f"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 186, 218])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f830fe44-0004-4968-88e2-304301a5a810, kind: Service { id: 060b7a36-3ed1-4e47-89cf-3f46bfb09b6c }, name: Name("nexus-060b7a36-3ed1-4e47-89cf-3f46bfb09b6c"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 157, 118])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
26992024-11-02T04:07:13.238ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27002024-11-02T04:07:13.238ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
27012024-11-02T04:07:13.238ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
27022024-11-02T04:07:13.238ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte0" rules = []
27032024-11-02T04:07:13.238ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27042024-11-02T04:07:13.238ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27052024-11-02T04:07:13.238ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 405 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:41290 req_id = 50df5b59-ef77-495e-90aa-a4e2afec1900 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
27062024-11-02T04:07:13.238ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/fda73319-9c6d-44d8-9377-b69537374c59/config/omicron-zones.json file = common/src/ledger.rs:177
27072024-11-02T04:07:13.467ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 229441 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:36945 req_id = a1b1a7c6-b7fe-4910-95cd-2df0195d61c4 response_code = 200 uri = /inventory
27082024-11-02T04:07:13.733ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:52715
27092024-11-02T04:07:13.733ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 92 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:52715 req_id = e556c95c-a8b2-40d9-bdb9-02d6754c322d response_code = 200 uri = /vpc-routes
27102024-11-02T04:07:13.775ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:59729
27112024-11-02T04:07:13.775ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 89 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:59729 req_id = 95a83e37-c71c-46e2-a8c1-a5fe81ea6e23 response_code = 200 uri = /vpc-routes
27122024-11-02T04:07:13.790ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
27132024-11-02T04:07:13.790ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 121 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:52715 req_id = eecc2236-7fc6-4f22-917f-5ef33dd297a6 response_code = 204 uri = /eip-gateways
27142024-11-02T04:07:13.814ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 251 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:52715 req_id = 5c8d6a7d-fc65-45e9-a1f8-3a78c69d0bc4 response_code = 204 uri = /vpc-routes
27152024-11-02T04:07:13.852ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
27162024-11-02T04:07:13.852ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 83 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:59729 req_id = 8ac276da-3fda-40d3-a676-cc54cfab89e8 response_code = 204 uri = /eip-gateways
27172024-11-02T04:07:13.862ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 173 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:59729 req_id = 6c606db9-ca75-4b7f-8a99-28f555a81aee response_code = 204 uri = /vpc-routes
27182024-11-02T04:07:13.978ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:40079
27192024-11-02T04:07:13.978ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
27202024-11-02T04:07:13.978ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
27212024-11-02T04:07:13.978ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/6308f1ec-c08e-4282-a1c2-db37d6b8a044/config/omicron-zones.json file = common/src/ledger.rs:177
27222024-11-02T04:07:13.978ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/fda73319-9c6d-44d8-9377-b69537374c59/config/omicron-zones.json file = common/src/ledger.rs:177
27232024-11-02T04:07:13.991ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:59598
27242024-11-02T04:07:13.996ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
27252024-11-02T04:07:14.049ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
27262024-11-02T04:07:14.132ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 141185 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:59598 req_id = 811cc846-2448-4444-b3cd-fd3524997743 response_code = 204 uri = /switch-ports
27272024-11-02T04:07:14.132ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:59009
27282024-11-02T04:07:14.133ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 157 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:59009 req_id = f88e9406-624c-4ea2-883e-5e2ffb8acafd response_code = 200 uri = /v2p
27292024-11-02T04:07:14.133ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 108 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:59598 req_id = f35f5b8a-de02-4757-b43b-b85479ab4dbc response_code = 200 uri = /network-bootstore-config
27302024-11-02T04:07:14.147ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:50482
27312024-11-02T04:07:14.147ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 162 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:50482 req_id = 6a43f635-713c-433f-bcad-44b7a896626a response_code = 200 uri = /v2p
27322024-11-02T04:07:14.200ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:34080
27332024-11-02T04:07:14.200ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
27342024-11-02T04:07:14.200ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
27352024-11-02T04:07:14.200ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/6308f1ec-c08e-4282-a1c2-db37d6b8a044/config/omicron-zones.json file = common/src/ledger.rs:177
27362024-11-02T04:07:14.202ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/fda73319-9c6d-44d8-9377-b69537374c59/config/omicron-zones.json file = common/src/ledger.rs:177
27372024-11-02T04:07:14.214ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:39277
27382024-11-02T04:07:14.215ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
27392024-11-02T04:07:14.256ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 277907 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:40079 req_id = 08dad531-c996-46b6-8217-907dad0f6a85 response_code = 200 uri = /inventory
27402024-11-02T04:07:14.262ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:36859
27412024-11-02T04:07:14.262ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:731 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 27d7c9ec-b8a8-4b31-ae12-c49ad36788f1, kind: Service { id: 7f79c18f-7b53-4b8a-86fc-bb23b923ea1d }, name: Name("external-dns-7f79c18f-7b53-4b8a-86fc-bb23b923ea1d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 238, 20])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 29b92946-7588-4b89-8147-322049e1f231, kind: Service { id: f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b }, name: Name("external-dns-f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 239, 77])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0cd6826d-84a0-4cd5-b7c6-32cb83627e6c, kind: Service { id: 9325209d-81b8-49e4-be6a-cedff3ef78c1 }, name: Name("nexus-9325209d-81b8-49e4-be6a-cedff3ef78c1"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 192, 104])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ec778b7a-3947-4776-98f2-be2ced817d66, kind: Service { id: d2bdfdcf-c904-463a-87e2-60b209fd743f }, name: Name("nexus-d2bdfdcf-c904-463a-87e2-60b209fd743f"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 186, 218])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f830fe44-0004-4968-88e2-304301a5a810, kind: Service { id: 060b7a36-3ed1-4e47-89cf-3f46bfb09b6c }, name: Name("nexus-060b7a36-3ed1-4e47-89cf-3f46bfb09b6c"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 157, 118])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
27422024-11-02T04:07:14.262ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27432024-11-02T04:07:14.262ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
27442024-11-02T04:07:14.262ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
27452024-11-02T04:07:14.262ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte0" rules = []
27462024-11-02T04:07:14.262ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27472024-11-02T04:07:14.263ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27482024-11-02T04:07:14.263ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 557 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:36859 req_id = e646ea25-8ffd-4d1f-8d63-3337312d0956 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
27492024-11-02T04:07:14.265ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
27502024-11-02T04:07:14.335ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 120120 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:39277 req_id = 64dfb28a-9d05-4167-ad68-5f695e73a319 response_code = 204 uri = /switch-ports
27512024-11-02T04:07:14.335ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 210 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:39277 req_id = 50bc8626-bb5e-420b-932a-f361b7318fb7 response_code = 200 uri = /network-bootstore-config
27522024-11-02T04:07:14.367ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:35524
27532024-11-02T04:07:14.368ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:731 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 27d7c9ec-b8a8-4b31-ae12-c49ad36788f1, kind: Service { id: 7f79c18f-7b53-4b8a-86fc-bb23b923ea1d }, name: Name("external-dns-7f79c18f-7b53-4b8a-86fc-bb23b923ea1d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 238, 20])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 29b92946-7588-4b89-8147-322049e1f231, kind: Service { id: f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b }, name: Name("external-dns-f3f0a2f4-1758-4c7a-a161-a4eb08bdc03b"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 239, 77])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0cd6826d-84a0-4cd5-b7c6-32cb83627e6c, kind: Service { id: 9325209d-81b8-49e4-be6a-cedff3ef78c1 }, name: Name("nexus-9325209d-81b8-49e4-be6a-cedff3ef78c1"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 192, 104])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ec778b7a-3947-4776-98f2-be2ced817d66, kind: Service { id: d2bdfdcf-c904-463a-87e2-60b209fd743f }, name: Name("nexus-d2bdfdcf-c904-463a-87e2-60b209fd743f"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 186, 218])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f830fe44-0004-4968-88e2-304301a5a810, kind: Service { id: 060b7a36-3ed1-4e47-89cf-3f46bfb09b6c }, name: Name("nexus-060b7a36-3ed1-4e47-89cf-3f46bfb09b6c"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 157, 118])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
27542024-11-02T04:07:14.368ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27552024-11-02T04:07:14.368ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
27562024-11-02T04:07:14.368ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
27572024-11-02T04:07:14.368ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte0" rules = []
27582024-11-02T04:07:14.368ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27592024-11-02T04:07:14.368ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27602024-11-02T04:07:14.368ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 323 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:35524 req_id = 1d9cd512-bc4f-4826-9a88-0afc3e1cfcf4 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
27612024-11-02T04:07:14.458ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 258202 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:34080 req_id = 24509caf-f5e0-4ca9-ae6b-b9e12826111f response_code = 200 uri = /inventory
27622024-11-02T04:07:15.412ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
27632024-11-02T04:07:15.413ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
27642024-11-02T04:07:15.413ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
27652024-11-02T04:07:17.306ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
27662024-11-02T04:07:17.307ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
27672024-11-02T04:07:17.308ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
27682024-11-02T04:07:17.313ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
27692024-11-02T04:07:17.342ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
27702024-11-02T04:07:17.349ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
27712024-11-02T04:07:17.349ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
27722024-11-02T04:07:17.350ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
27732024-11-02T04:07:17.350ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
27742024-11-02T04:07:17.351ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
27752024-11-02T04:07:17.355ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
27762024-11-02T04:07:17.357ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
27772024-11-02T04:07:17.358ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
27782024-11-02T04:07:17.400ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
27792024-11-02T04:07:17.410ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
27802024-11-02T04:07:25.416ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
27812024-11-02T04:07:25.417ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
27822024-11-02T04:07:25.418ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
27832024-11-02T04:07:27.310ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
27842024-11-02T04:07:27.311ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
27852024-11-02T04:07:27.311ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
27862024-11-02T04:07:27.314ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
27872024-11-02T04:07:27.343ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
27882024-11-02T04:07:27.350ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
27892024-11-02T04:07:27.351ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
27902024-11-02T04:07:27.352ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
27912024-11-02T04:07:27.352ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
27922024-11-02T04:07:27.353ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
27932024-11-02T04:07:27.356ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
27942024-11-02T04:07:27.360ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
27952024-11-02T04:07:27.360ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
27962024-11-02T04:07:27.401ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
27972024-11-02T04:07:27.411ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
27982024-11-02T04:07:30.394ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:61608
27992024-11-02T04:07:30.394ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 51 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:61608 req_id = ee6c960f-246a-45ef-a79e-cbc13e117874 response_code = 200 uri = /baseboard
28002024-11-02T04:07:35.421ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28012024-11-02T04:07:35.422ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28022024-11-02T04:07:35.422ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28032024-11-02T04:07:37.314ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28042024-11-02T04:07:37.315ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28052024-11-02T04:07:37.316ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28062024-11-02T04:07:37.316ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28072024-11-02T04:07:37.345ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28082024-11-02T04:07:37.352ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28092024-11-02T04:07:37.353ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28102024-11-02T04:07:37.354ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28112024-11-02T04:07:37.354ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28122024-11-02T04:07:37.355ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28132024-11-02T04:07:37.357ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28142024-11-02T04:07:37.361ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28152024-11-02T04:07:37.362ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28162024-11-02T04:07:37.403ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28172024-11-02T04:07:37.412ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28182024-11-02T04:07:42.838ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:41924
28192024-11-02T04:07:42.839ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 85 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:41924 req_id = a09b849a-8206-4965-9fc8-44f25a2e83a1 response_code = 200 uri = /vpc-routes
28202024-11-02T04:07:42.881ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
28212024-11-02T04:07:42.881ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 58 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:41924 req_id = fe96ea34-e754-4cd5-9ace-03d6116c8325 response_code = 204 uri = /eip-gateways
28222024-11-02T04:07:42.889ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 225 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:41924 req_id = c79e17a2-789a-4a62-a63a-c637973612ba response_code = 204 uri = /vpc-routes
28232024-11-02T04:07:42.896ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:37038
28242024-11-02T04:07:42.896ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
28252024-11-02T04:07:42.939ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
28262024-11-02T04:07:42.999ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 103244 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:37038 req_id = b0d645d1-a1c9-4b2a-9e4e-3f8f77c367b4 response_code = 204 uri = /switch-ports
28272024-11-02T04:07:42.999ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:54707
28282024-11-02T04:07:42.999ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 242 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:37038 req_id = 5037737c-dbc8-4b66-93e4-bda182b304d9 response_code = 200 uri = /network-bootstore-config
28292024-11-02T04:07:42.999ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 219 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:54707 req_id = 74bedfb0-b47b-4051-8b6b-f52bbc214c74 response_code = 200 uri = /v2p
28302024-11-02T04:07:43.634ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:45778
28312024-11-02T04:07:43.634ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 92 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:45778 req_id = 923100ee-f634-4ddb-b9fe-6a94c540daa1 response_code = 200 uri = /vpc-routes
28322024-11-02T04:07:43.691ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
28332024-11-02T04:07:43.691ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 92 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:45778 req_id = 1c33d269-6d67-4e48-9bb8-78b6da134cc1 response_code = 204 uri = /eip-gateways
28342024-11-02T04:07:43.730ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:41161
28352024-11-02T04:07:43.730ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 177 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:41161 req_id = bff13232-183f-4ba3-b2e7-7263e669804c response_code = 200 uri = /v2p
28362024-11-02T04:07:43.733ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 185 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:45778 req_id = b6b5833f-1f20-447e-8557-7c238adf6866 response_code = 204 uri = /vpc-routes
28372024-11-02T04:07:43.750ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:64201
28382024-11-02T04:07:43.750ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 31 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:64201 req_id = 280d31b0-5b36-4374-b6f4-a25177542bc8 response_code = 200 uri = /vpc-routes
28392024-11-02T04:07:43.765ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
28402024-11-02T04:07:43.765ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 40 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:64201 req_id = 511f1f70-525d-4da3-bda6-a42b5c882cec response_code = 204 uri = /eip-gateways
28412024-11-02T04:07:43.774ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 174 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:64201 req_id = 847c0e40-8ff1-41b1-9205-d2f0cd073d29 response_code = 204 uri = /vpc-routes
28422024-11-02T04:07:43.778ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:54692
28432024-11-02T04:07:43.778ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
28442024-11-02T04:07:43.821ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
28452024-11-02T04:07:43.883ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 104753 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:54692 req_id = fd10caa3-fbd4-4dbd-80aa-6fbdb0bc1663 response_code = 204 uri = /switch-ports
28462024-11-02T04:07:43.883ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:41772
28472024-11-02T04:07:43.883ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 106 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:41772 req_id = 79367f48-edde-47d0-a47f-9f389d30ee9b response_code = 200 uri = /v2p
28482024-11-02T04:07:43.883ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 88 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:54692 req_id = 7559f3c1-6e60-448f-9103-b084f0e1bcb9 response_code = 200 uri = /network-bootstore-config
28492024-11-02T04:07:43.913ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:65289
28502024-11-02T04:07:43.913ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
28512024-11-02T04:07:43.955ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
28522024-11-02T04:07:44.016ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 103002 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:65289 req_id = 9636d6b6-d34c-4116-a96b-4e391d029e29 response_code = 204 uri = /switch-ports
28532024-11-02T04:07:44.016ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 106 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:65289 req_id = 93c98fbb-48ad-4b41-922a-2ab860aa4075 response_code = 200 uri = /network-bootstore-config
28542024-11-02T04:07:45.425ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28552024-11-02T04:07:45.426ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28562024-11-02T04:07:45.426ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28572024-11-02T04:07:47.318ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28582024-11-02T04:07:47.319ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28592024-11-02T04:07:47.320ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28602024-11-02T04:07:47.320ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28612024-11-02T04:07:47.348ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28622024-11-02T04:07:47.354ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28632024-11-02T04:07:47.355ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28642024-11-02T04:07:47.356ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28652024-11-02T04:07:47.356ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28662024-11-02T04:07:47.357ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28672024-11-02T04:07:47.360ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28682024-11-02T04:07:47.364ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28692024-11-02T04:07:47.364ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28702024-11-02T04:07:47.405ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28712024-11-02T04:07:47.415ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28722024-11-02T04:07:55.429ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28732024-11-02T04:07:55.429ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28742024-11-02T04:07:55.430ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28752024-11-02T04:07:57.322ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28762024-11-02T04:07:57.323ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28772024-11-02T04:07:57.324ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28782024-11-02T04:07:57.324ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28792024-11-02T04:07:57.350ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28802024-11-02T04:07:57.357ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28812024-11-02T04:07:57.358ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28822024-11-02T04:07:57.358ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28832024-11-02T04:07:57.359ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28842024-11-02T04:07:57.359ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28852024-11-02T04:07:57.361ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28862024-11-02T04:07:57.365ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28872024-11-02T04:07:57.366ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28882024-11-02T04:07:57.407ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28892024-11-02T04:07:57.416ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28902024-11-02T04:08:00.427ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:60908
28912024-11-02T04:08:00.428ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 72 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:60908 req_id = ecb70058-e1a2-4770-8a3f-3fe0a4a606b0 response_code = 200 uri = /baseboard
28922024-11-02T04:08:05.433ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28932024-11-02T04:08:05.433ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28942024-11-02T04:08:05.434ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28952024-11-02T04:08:07.327ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28962024-11-02T04:08:07.327ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28972024-11-02T04:08:07.328ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28982024-11-02T04:08:07.329ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
28992024-11-02T04:08:07.352ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29002024-11-02T04:08:07.359ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29012024-11-02T04:08:07.360ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29022024-11-02T04:08:07.360ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29032024-11-02T04:08:07.361ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29042024-11-02T04:08:07.361ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29052024-11-02T04:08:07.363ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29062024-11-02T04:08:07.368ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29072024-11-02T04:08:07.368ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29082024-11-02T04:08:07.409ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29092024-11-02T04:08:07.419ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29102024-11-02T04:08:12.974ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:40820
29112024-11-02T04:08:12.974ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 73 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:40820 req_id = a6b38ffa-0ba2-4a75-9f64-9c99ca70a589 response_code = 200 uri = /vpc-routes
29122024-11-02T04:08:12.995ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:56906
29132024-11-02T04:08:12.995ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 135 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:56906 req_id = ac3975eb-874e-4eaa-aa0a-0f7521f371f4 response_code = 200 uri = /v2p
29142024-11-02T04:08:13.009ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
29152024-11-02T04:08:13.009ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 90 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:40820 req_id = 2e2607d9-06b5-44e8-9473-b599ae8f23a5 response_code = 204 uri = /eip-gateways
29162024-11-02T04:08:13.017ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 195 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:40820 req_id = 70c8d617-72b8-48eb-8894-734e200dc00d response_code = 204 uri = /vpc-routes
29172024-11-02T04:08:13.248ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:41452
29182024-11-02T04:08:13.248ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
29192024-11-02T04:08:13.291ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
29202024-11-02T04:08:13.354ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 105967 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:41452 req_id = aab0f20e-cdb4-4b97-8344-a7365ae6d2b1 response_code = 204 uri = /switch-ports
29212024-11-02T04:08:13.355ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 92 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:41452 req_id = ac526949-831d-424a-9d11-f7c8c41085e4 response_code = 200 uri = /network-bootstore-config
29222024-11-02T04:08:13.621ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:50658
29232024-11-02T04:08:13.621ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 80 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:50658 req_id = e52a343a-b9b8-47d2-9cc7-ae4d5aed3854 response_code = 200 uri = /vpc-routes
29242024-11-02T04:08:13.710ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
29252024-11-02T04:08:13.710ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 117 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:50658 req_id = cc4afbe6-57ff-41e5-8bdb-ba944274be08 response_code = 204 uri = /eip-gateways
29262024-11-02T04:08:13.720ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 255 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:50658 req_id = 7a661185-0691-4175-a188-e3d40a9da21b response_code = 204 uri = /vpc-routes
29272024-11-02T04:08:13.834ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:48062
29282024-11-02T04:08:13.834ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 128 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:48062 req_id = 931cea6d-2403-4174-a634-32bdafa6719a response_code = 200 uri = /v2p
29292024-11-02T04:08:13.835ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:59981
29302024-11-02T04:08:13.835ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 33 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:59981 req_id = 330f2c98-27fb-437b-8de5-a890dff1bffe response_code = 200 uri = /vpc-routes
29312024-11-02T04:08:13.880ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:65275
29322024-11-02T04:08:13.880ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
29332024-11-02T04:08:13.923ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
29342024-11-02T04:08:13.989ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 108524 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:65275 req_id = 2534b283-daa3-4a8f-bbad-273ab7030768 response_code = 204 uri = /switch-ports
29352024-11-02T04:08:13.989ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
29362024-11-02T04:08:13.989ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 61 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:59981 req_id = 06a320c3-98d5-464a-9e3e-06e6f979fa33 response_code = 204 uri = /eip-gateways
29372024-11-02T04:08:13.989ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 86 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:65275 req_id = 60b931ec-531e-4265-8242-65b62f569627 response_code = 200 uri = /network-bootstore-config
29382024-11-02T04:08:14.100ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:36683
29392024-11-02T04:08:14.100ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 168 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:36683 req_id = 91a9bf6d-65d1-4142-836d-49fe6a01b8bf response_code = 200 uri = /v2p
29402024-11-02T04:08:14.107ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 193 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:59981 req_id = 18fe94aa-1281-4b9e-9665-af5a42eb5b87 response_code = 204 uri = /vpc-routes
29412024-11-02T04:08:14.156ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:49480
29422024-11-02T04:08:14.156ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
29432024-11-02T04:08:14.197ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
29442024-11-02T04:08:14.257ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 100921 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:49480 req_id = 8c665e14-75b1-447a-beca-124da8ccd93e response_code = 204 uri = /switch-ports
29452024-11-02T04:08:14.258ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 110 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:49480 req_id = fdebea32-b741-4848-b0b7-3e8a2b9f336c response_code = 200 uri = /network-bootstore-config
29462024-11-02T04:08:15.436ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29472024-11-02T04:08:15.437ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29482024-11-02T04:08:15.438ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29492024-11-02T04:08:17.331ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29502024-11-02T04:08:17.331ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29512024-11-02T04:08:17.332ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29522024-11-02T04:08:17.333ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29532024-11-02T04:08:17.354ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29542024-11-02T04:08:17.360ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29552024-11-02T04:08:17.361ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29562024-11-02T04:08:17.362ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29572024-11-02T04:08:17.362ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29582024-11-02T04:08:17.363ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29592024-11-02T04:08:17.365ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29602024-11-02T04:08:17.369ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29612024-11-02T04:08:17.370ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29622024-11-02T04:08:17.411ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29632024-11-02T04:08:17.421ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29642024-11-02T04:08:25.441ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29652024-11-02T04:08:25.442ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29662024-11-02T04:08:25.443ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29672024-11-02T04:08:27.335ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29682024-11-02T04:08:27.336ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29692024-11-02T04:08:27.336ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29702024-11-02T04:08:27.337ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29712024-11-02T04:08:27.356ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29722024-11-02T04:08:27.362ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29732024-11-02T04:08:27.363ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29742024-11-02T04:08:27.363ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29752024-11-02T04:08:27.365ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29762024-11-02T04:08:27.365ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29772024-11-02T04:08:27.366ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29782024-11-02T04:08:27.372ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29792024-11-02T04:08:27.373ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29802024-11-02T04:08:27.413ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29812024-11-02T04:08:27.422ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29822024-11-02T04:08:30.461ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:61804
29832024-11-02T04:08:30.462ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 52 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:61804 req_id = 697a6fc1-0b6d-4207-a850-00a8f11e1739 response_code = 200 uri = /baseboard
29842024-11-02T04:08:35.445ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29852024-11-02T04:08:35.445ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29862024-11-02T04:08:35.446ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29872024-11-02T04:08:37.339ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29882024-11-02T04:08:37.340ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29892024-11-02T04:08:37.340ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29902024-11-02T04:08:37.341ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29912024-11-02T04:08:37.359ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29922024-11-02T04:08:37.364ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29932024-11-02T04:08:37.364ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29942024-11-02T04:08:37.365ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29952024-11-02T04:08:37.366ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29962024-11-02T04:08:37.367ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29972024-11-02T04:08:37.368ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29982024-11-02T04:08:37.374ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
29992024-11-02T04:08:37.374ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30002024-11-02T04:08:37.415ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30012024-11-02T04:08:37.425ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30022024-11-02T04:08:42.930ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:65007
30032024-11-02T04:08:42.930ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 94 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:65007 req_id = 7619c1a0-4c9b-4963-8be5-f15f64c73266 response_code = 200 uri = /vpc-routes
30042024-11-02T04:08:42.946ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
30052024-11-02T04:08:42.946ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 69 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:65007 req_id = f96a2e10-9b65-499b-8172-ccdefbf7a3ff response_code = 204 uri = /eip-gateways
30062024-11-02T04:08:42.953ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 201 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:65007 req_id = 875ba30a-3cf5-471b-bbae-45aeee8852fd response_code = 204 uri = /vpc-routes
30072024-11-02T04:08:43.024ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:40383
30082024-11-02T04:08:43.024ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
30092024-11-02T04:08:43.070ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
30102024-11-02T04:08:43.129ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 105182 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:40383 req_id = 38c8dc34-20b8-45eb-84a6-df7129580580 response_code = 204 uri = /switch-ports
30112024-11-02T04:08:43.130ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:38639
30122024-11-02T04:08:43.130ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 116 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:40383 req_id = 2aaadd41-a2e4-4fde-af7a-f09f297bb4f6 response_code = 200 uri = /network-bootstore-config
30132024-11-02T04:08:43.130ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 134 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:38639 req_id = e56807bc-8245-4d9b-9322-fd9578275c17 response_code = 200 uri = /v2p
30142024-11-02T04:08:43.631ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:54259
30152024-11-02T04:08:43.631ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 84 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:54259 req_id = 6808bf80-55d7-495e-a904-cef137a239a1 response_code = 200 uri = /vpc-routes
30162024-11-02T04:08:43.646ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
30172024-11-02T04:08:43.646ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 113 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:54259 req_id = aeb05eb3-4893-4ccd-b7e3-0048bebaff52 response_code = 204 uri = /eip-gateways
30182024-11-02T04:08:43.693ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:50015
30192024-11-02T04:08:43.694ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 173 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:50015 req_id = 3026a9c4-4ebd-4db2-ba7c-47b6c2824458 response_code = 200 uri = /v2p
30202024-11-02T04:08:43.704ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 237 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:54259 req_id = f4ca5de8-30ec-4637-bb22-be61f83399a7 response_code = 204 uri = /vpc-routes
30212024-11-02T04:08:43.752ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:49552
30222024-11-02T04:08:43.752ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
30232024-11-02T04:08:43.795ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
30242024-11-02T04:08:43.857ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 105054 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:49552 req_id = 52a6e430-363f-4f71-8409-0d37036719c8 response_code = 204 uri = /switch-ports
30252024-11-02T04:08:43.857ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:42247
30262024-11-02T04:08:43.858ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 61 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:42247 req_id = 2d90324b-13ca-4115-b198-0389ab85718b response_code = 200 uri = /vpc-routes
30272024-11-02T04:08:43.858ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 87 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:49552 req_id = c3227460-c215-497c-b4bf-449b3cdb370f response_code = 200 uri = /network-bootstore-config
30282024-11-02T04:08:43.873ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
30292024-11-02T04:08:43.873ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 105 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:42247 req_id = 20435fc1-36c1-421b-b9f2-9db4524b77f7 response_code = 204 uri = /eip-gateways
30302024-11-02T04:08:43.881ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 212 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:42247 req_id = d318c715-a797-44ef-809c-4cc35e94b093 response_code = 204 uri = /vpc-routes
30312024-11-02T04:08:43.906ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:57713
30322024-11-02T04:08:43.906ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 125 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:57713 req_id = 460ba2b7-0aa3-4314-8783-649e25c4d419 response_code = 200 uri = /v2p
30332024-11-02T04:08:43.933ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51785
30342024-11-02T04:08:43.933ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
30352024-11-02T04:08:43.975ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
30362024-11-02T04:08:44.043ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 109676 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:51785 req_id = c185985d-71a0-457e-8bc4-f215a0aa3666 response_code = 204 uri = /switch-ports
30372024-11-02T04:08:44.044ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 193 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:51785 req_id = e75fa71d-98de-4f6d-a3ce-72cc833ec98a response_code = 200 uri = /network-bootstore-config
30382024-11-02T04:08:45.449ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30392024-11-02T04:08:45.449ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30402024-11-02T04:08:45.450ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30412024-11-02T04:08:47.343ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30422024-11-02T04:08:47.344ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30432024-11-02T04:08:47.345ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30442024-11-02T04:08:47.345ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30452024-11-02T04:08:47.360ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30462024-11-02T04:08:47.366ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30472024-11-02T04:08:47.367ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30482024-11-02T04:08:47.367ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30492024-11-02T04:08:47.369ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30502024-11-02T04:08:47.369ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30512024-11-02T04:08:47.370ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30522024-11-02T04:08:47.376ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30532024-11-02T04:08:47.376ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30542024-11-02T04:08:47.417ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30552024-11-02T04:08:47.427ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30562024-11-02T04:08:55.453ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30572024-11-02T04:08:55.454ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30582024-11-02T04:08:55.454ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30592024-11-02T04:08:57.348ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30602024-11-02T04:08:57.349ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30612024-11-02T04:08:57.349ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30622024-11-02T04:08:57.350ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30632024-11-02T04:08:57.362ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30642024-11-02T04:08:57.368ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30652024-11-02T04:08:57.369ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30662024-11-02T04:08:57.369ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30672024-11-02T04:08:57.371ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30682024-11-02T04:08:57.372ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30692024-11-02T04:08:57.372ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30702024-11-02T04:08:57.378ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30712024-11-02T04:08:57.379ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30722024-11-02T04:08:57.419ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30732024-11-02T04:08:57.429ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30742024-11-02T04:09:00.496ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:58218
30752024-11-02T04:09:00.496ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 58 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:58218 req_id = 2f18cc22-aeaa-422a-b918-95eb87330522 response_code = 200 uri = /baseboard
30762024-11-02T04:09:05.457ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30772024-11-02T04:09:05.457ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30782024-11-02T04:09:05.458ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30792024-11-02T04:09:07.352ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30802024-11-02T04:09:07.353ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30812024-11-02T04:09:07.353ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30822024-11-02T04:09:07.354ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30832024-11-02T04:09:07.364ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30842024-11-02T04:09:07.369ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30852024-11-02T04:09:07.370ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30862024-11-02T04:09:07.371ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30872024-11-02T04:09:07.372ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30882024-11-02T04:09:07.374ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30892024-11-02T04:09:07.375ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30902024-11-02T04:09:07.380ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30912024-11-02T04:09:07.381ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30922024-11-02T04:09:07.421ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30932024-11-02T04:09:07.431ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
30942024-11-02T04:09:12.977ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:41310
30952024-11-02T04:09:12.977ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 58 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:41310 req_id = b3412dc0-071b-45fc-b58b-2514f202d664 response_code = 200 uri = /vpc-routes
30962024-11-02T04:09:12.995ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
30972024-11-02T04:09:12.996ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 86 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:41310 req_id = 1848eb16-0b69-4b12-8ca8-6e2deaaba403 response_code = 204 uri = /eip-gateways
30982024-11-02T04:09:13.003ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 232 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:41310 req_id = 1fb4e67a-87ac-46d7-a0ec-bdc4efe8fee5 response_code = 204 uri = /vpc-routes
30992024-11-02T04:09:13.230ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:59402
31002024-11-02T04:09:13.230ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 169 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:59402 req_id = 74d62516-a1f2-4114-83fc-f77f34f05d2a response_code = 200 uri = /v2p
31012024-11-02T04:09:13.279ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:37202
31022024-11-02T04:09:13.279ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
31032024-11-02T04:09:13.324ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
31042024-11-02T04:09:13.392ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 113778 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:37202 req_id = 44722af2-9e5a-404b-83d9-100a93e0af30 response_code = 204 uri = /switch-ports
31052024-11-02T04:09:13.393ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 102 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:37202 req_id = 32772296-5c65-467d-b507-e8bffab13313 response_code = 200 uri = /network-bootstore-config
31062024-11-02T04:09:13.747ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:37474
31072024-11-02T04:09:13.747ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 89 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:37474 req_id = a9c0d3a4-a095-4796-8407-6980da6eb947 response_code = 200 uri = /vpc-routes
31082024-11-02T04:09:13.762ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
31092024-11-02T04:09:13.762ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 68 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:37474 req_id = 9ae08fc2-01da-41bc-a859-73ce5b1df183 response_code = 204 uri = /eip-gateways
31102024-11-02T04:09:13.771ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 279 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:37474 req_id = e1e800ab-5e1e-4bd7-9ffa-15feaae4a744 response_code = 204 uri = /vpc-routes
31112024-11-02T04:09:13.844ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:36027
31122024-11-02T04:09:13.844ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 123 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:36027 req_id = d7eae97c-189f-4e9a-b4f1-4d5ca055421c response_code = 200 uri = /v2p
31132024-11-02T04:09:13.868ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:50414
31142024-11-02T04:09:13.868ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
31152024-11-02T04:09:13.915ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
31162024-11-02T04:09:13.983ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 115402 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:50414 req_id = b5305f9d-1b35-4896-a565-181a9060c9df response_code = 204 uri = /switch-ports
31172024-11-02T04:09:13.983ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:59085
31182024-11-02T04:09:13.983ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:37321
31192024-11-02T04:09:13.984ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 143 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:50414 req_id = 90ce337c-5605-4b65-9391-0f8b1c372df4 response_code = 200 uri = /network-bootstore-config
31202024-11-02T04:09:13.984ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 127 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:59085 req_id = 55ad1c8d-3307-467f-bc39-5417fd704198 response_code = 200 uri = /vpc-routes
31212024-11-02T04:09:13.984ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 248 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:37321 req_id = a90a7f94-091e-4478-883d-182645adfa9d response_code = 200 uri = /v2p
31222024-11-02T04:09:14.042ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
31232024-11-02T04:09:14.042ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 93 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:59085 req_id = 703c27dd-d67a-490f-a205-7eadb19f3038 response_code = 204 uri = /eip-gateways
31242024-11-02T04:09:14.052ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 215 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:59085 req_id = 4350ca9f-ce2e-4fc8-a52d-f8b64683b7fd response_code = 204 uri = /vpc-routes
31252024-11-02T04:09:14.121ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:60586
31262024-11-02T04:09:14.121ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4081
31272024-11-02T04:09:14.168ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4105
31282024-11-02T04:09:14.241ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 119403 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:60586 req_id = 547da329-5809-487b-96e5-93a5c2b29b40 response_code = 204 uri = /switch-ports
31292024-11-02T04:09:14.241ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 101 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:60586 req_id = b9b1d966-3755-4101-acc0-51b537c9e2ff response_code = 200 uri = /network-bootstore-config
31302024-11-02T04:09:14.967ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:36899
31312024-11-02T04:09:14.967ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 169 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:36899 req_id = c30821ba-3339-440f-8041-62de35e51cba response_code = 200 uri = /v2p
31322024-11-02T04:09:14.968ZINFOSledAgent (PortManager): Mapping virtual NIC to physical host file = illumos-utils/src/opte/port_manager.rs:838 mapping = VirtualNetworkInterfaceHost { virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), physical_host_ip: fd00:1122:3344:101::1, vni: Vni(5808556) }
31332024-11-02T04:09:14.968ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 101 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:36899 req_id = b3b97e7d-7a53-46ae-9df4-e8845e79222d response_code = 204 uri = /v2p
31342024-11-02T04:09:15.101ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:65140
31352024-11-02T04:09:15.107ZINFOSledAgent (InstanceManager): ensuring instance is registered file = sled-agent/src/instance_manager.rs:555 hardware = InstanceHardware { properties: InstanceProperties { ncpus: InstanceCpuCount(2), memory: ByteCount(1073741824), hostname: Hostname("localshark") }, nics: [NetworkInterface { id: 85f666a4-82d8-4fb1-89a8-06014ddd431b, kind: Instance { id: 70aeba8a-4066-4620-bb46-a15fa016442a }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(5808556), primary: true, slot: 0, transit_ips: [] }], source_nat: SourceNatConfig { ip: 10.151.1.175, first_port: 0, last_port: 16383 }, ephemeral_ip: Some(10.151.1.176), floating_ips: [], firewall_rules: [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 85f666a4-82d8-4fb1-89a8-06014ddd431b, kind: Instance { id: 70aeba8a-4066-4620-bb46-a15fa016442a }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(5808556), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 85f666a4-82d8-4fb1-89a8-06014ddd431b, kind: Instance { id: 70aeba8a-4066-4620-bb46-a15fa016442a }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(5808556), primary: true, slot: 0, transit_ips: [] }], filter_hosts: Some([Vpc(Vni(5808556))]), filter_ports: None, filter_protocols: None, action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 85f666a4-82d8-4fb1-89a8-06014ddd431b, kind: Instance { id: 70aeba8a-4066-4620-bb46-a15fa016442a }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(5808556), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(22), last: L4Port(22) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }], dhcp_config: DhcpConfig { dns_servers: [1.1.1.1, 9.9.9.9], host_domain: None, search_domains: [] }, disks: [DiskRequest { device: "nvme", name: "disk-a1a8c70c502", read_only: false, slot: Slot(0), volume_construction_request: Volume { block_size: 512, id: afd8d229-26fe-4bad-aba3-9facf6500bfe, read_only_parent: Some(Volume { block_size: 512, id: fcac098c-3953-4a98-a686-5b15dd9c7d30, read_only_parent: None, sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 2, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 606b646d-156a-4ea2-9c58-96cd3d861017, key: Some("7zjTQ4fe6XG8fFsNxGFcQsOjWmwMbLlk2ok3Xv2QlEc="), key_pem: None, lossy: false, read_only: true, root_cert_pem: None, target: ["[fd00:1122:3344:101::14]:19002", "[fd00:1122:3344:101::13]:19001", "[fd00:1122:3344:101::16]:19001"] } }] }), sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 1, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: afd8d229-26fe-4bad-aba3-9facf6500bfe, key: Some("c2oKxUAz4F+jZE/gP+XuwO6qt61P/bxKXhaw4AmB8D4="), key_pem: None, lossy: false, read_only: false, root_cert_pem: None, target: ["[fd00:1122:3344:101::15]:19001", "[fd00:1122:3344:101::14]:19003", "[fd00:1122:3344:101::16]:19002"] } }] } }], boot_settings: Some(BootSettings { order: [BootOrderEntry { name: "disk-a1a8c70c502" }] }), cloud_init_bytes: Some(..) } instance_id = 70aeba8a-4066-4620-bb46-a15fa016442a metadata = InstanceMetadata { silo_id: 73ce13ac-7ea3-466e-9d78-11e4419419eb, project_id: 453d9a68-e399-48d9-9ee3-b25d4e917610 } migration_id = None propolis_addr = [fd00:1122:3344:101::1:0]:12400 propolis_id = ae2b4e8c-8f07-47d2-b46e-f505dadf6695 vmm_runtime = VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2024-11-02T04:09:15.100901160Z }
31362024-11-02T04:09:15.107ZINFOSledAgent (InstanceManager): registering new instance file = sled-agent/src/instance_manager.rs:588 instance_id = 70aeba8a-4066-4620-bb46-a15fa016442a migration_id = None propolis_id = ae2b4e8c-8f07-47d2-b46e-f505dadf6695
31372024-11-02T04:09:15.107ZINFOSledAgent (InstanceManager): initializing new Instance file = sled-agent/src/instance.rs:1281 instance_id = 70aeba8a-4066-4620-bb46-a15fa016442a migration_id = None propolis_id = ae2b4e8c-8f07-47d2-b46e-f505dadf6695 state = InstanceInitialState { hardware: InstanceHardware { properties: InstanceProperties { ncpus: InstanceCpuCount(2), memory: ByteCount(1073741824), hostname: Hostname("localshark") }, nics: [NetworkInterface { id: 85f666a4-82d8-4fb1-89a8-06014ddd431b, kind: Instance { id: 70aeba8a-4066-4620-bb46-a15fa016442a }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(5808556), primary: true, slot: 0, transit_ips: [] }], source_nat: SourceNatConfig { ip: 10.151.1.175, first_port: 0, last_port: 16383 }, ephemeral_ip: Some(10.151.1.176), floating_ips: [], firewall_rules: [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 85f666a4-82d8-4fb1-89a8-06014ddd431b, kind: Instance { id: 70aeba8a-4066-4620-bb46-a15fa016442a }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(5808556), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 85f666a4-82d8-4fb1-89a8-06014ddd431b, kind: Instance { id: 70aeba8a-4066-4620-bb46-a15fa016442a }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(5808556), primary: true, slot: 0, transit_ips: [] }], filter_hosts: Some([Vpc(Vni(5808556))]), filter_ports: None, filter_protocols: None, action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 85f666a4-82d8-4fb1-89a8-06014ddd431b, kind: Instance { id: 70aeba8a-4066-4620-bb46-a15fa016442a }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(5808556), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(22), last: L4Port(22) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }], dhcp_config: DhcpConfig { dns_servers: [1.1.1.1, 9.9.9.9], host_domain: None, search_domains: [] }, disks: [DiskRequest { device: "nvme", name: "disk-a1a8c70c502", read_only: false, slot: Slot(0), volume_construction_request: Volume { block_size: 512, id: afd8d229-26fe-4bad-aba3-9facf6500bfe, read_only_parent: Some(Volume { block_size: 512, id: fcac098c-3953-4a98-a686-5b15dd9c7d30, read_only_parent: None, sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 2, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 606b646d-156a-4ea2-9c58-96cd3d861017, key: Some("7zjTQ4fe6XG8fFsNxGFcQsOjWmwMbLlk2ok3Xv2QlEc="), key_pem: None, lossy: false, read_only: true, root_cert_pem: None, target: ["[fd00:1122:3344:101::14]:19002", "[fd00:1122:3344:101::13]:19001", "[fd00:1122:3344:101::16]:19001"] } }] }), sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 1, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: afd8d229-26fe-4bad-aba3-9facf6500bfe, key: Some("c2oKxUAz4F+jZE/gP+XuwO6qt61P/bxKXhaw4AmB8D4="), key_pem: None, lossy: false, read_only: false, root_cert_pem: None, target: ["[fd00:1122:3344:101::15]:19001", "[fd00:1122:3344:101::14]:19003", "[fd00:1122:3344:101::16]:19002"] } }] } }], boot_settings: Some(BootSettings { order: [BootOrderEntry { name: "disk-a1a8c70c502" }] }), cloud_init_bytes: Some(..) }, vmm_runtime: VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2024-11-02T04:09:15.100901160Z }, propolis_addr: [fd00:1122:3344:101::1:0]:12400, migration_id: None }
31382024-11-02T04:09:15.108ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 7052 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:65140 req_id = ed29b91b-6585-41c1-a5f0-41a94981871f response_code = 200 uri = /vmms/ae2b4e8c-8f07-47d2-b46e-f505dadf6695
31392024-11-02T04:09:15.185ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:51634
31402024-11-02T04:09:15.186ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 896 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:51634 req_id = dbba820c-d0f3-4953-9d92-5914f591f3f5 response_code = 200 uri = /vmms/ae2b4e8c-8f07-47d2-b46e-f505dadf6695/state
31412024-11-02T04:09:15.188ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:430 port = Port { inner: PortInner(PortData { name: "opte6", ip: 172.30.0.5, mac: MacAddr6([168, 64, 37, 240, 0, 0]), slot: 0, vni: Vni { inner: 5808556 }, subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.0.1 } }) }
31422024-11-02T04:09:15.188ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
31432024-11-02T04:09:15.231ZINFOSledAgent (InstanceManager): Configuring new Omicron zone: oxz_propolis-server_ae2b4e8c-8f07-47d2-b46e-f505dadf6695 file = illumos-utils/src/zone.rs:325 instance_id = 70aeba8a-4066-4620-bb46-a15fa016442a propolis_id = ae2b4e8c-8f07-47d2-b46e-f505dadf6695
31442024-11-02T04:09:15.253ZINFOSledAgent (InstanceManager): Installing Omicron zone: oxz_propolis-server_ae2b4e8c-8f07-47d2-b46e-f505dadf6695 file = illumos-utils/src/zone.rs:363 instance_id = 70aeba8a-4066-4620-bb46-a15fa016442a propolis_id = ae2b4e8c-8f07-47d2-b46e-f505dadf6695
31452024-11-02T04:09:15.288ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:57096
31462024-11-02T04:09:15.289ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 108 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:57096 req_id = f3fba4df-1312-4698-a4ed-6f74c4da7a4c response_code = 200 uri = /vpc-routes
31472024-11-02T04:09:15.307ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {85f666a4-82d8-4fb1-89a8-06014ddd431b: {10.151.1.175: {33dd95f9-7f30-4ff5-91b2-6525e227cc31}, 10.151.1.176: {33dd95f9-7f30-4ff5-91b2-6525e227cc31}}} }
31482024-11-02T04:09:15.463ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
31492024-11-02T04:09:15.464ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
31502024-11-02T04:09:15.465ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
31512024-11-02T04:09:17.001ZINFOSledAgent (InstanceManager): Profile for oxz_propolis-server_ae2b4e8c-8f07-47d2-b46e-f505dadf6695: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="system/illumos/propolis-server"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlInstance0'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::1:0'/> <propval type="astring" name="listen_port" value='12400'/> <propval type="astring" name="metric_addr" value='dns'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34 instance_id = 70aeba8a-4066-4620-bb46-a15fa016442a propolis_id = ae2b4e8c-8f07-47d2-b46e-f505dadf6695
31522024-11-02T04:09:17.001ZINFOSledAgent (InstanceManager): Booting oxz_propolis-server_ae2b4e8c-8f07-47d2-b46e-f505dadf6695 zone file = illumos-utils/src/running_zone.rs:488 instance_id = 70aeba8a-4066-4620-bb46-a15fa016442a propolis_id = ae2b4e8c-8f07-47d2-b46e-f505dadf6695 zone = oxz_propolis-server_ae2b4e8c-8f07-47d2-b46e-f505dadf6695
31532024-11-02T04:09:17.190ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_ae2b4e8c-8f07-47d2-b46e-f505dadf6695") failed: Property not found. retry in 66.835699ms file = illumos-utils/src/svc.rs:36 instance_id = 70aeba8a-4066-4620-bb46-a15fa016442a propolis_id = ae2b4e8c-8f07-47d2-b46e-f505dadf6695 zone = oxz_propolis-server_ae2b4e8c-8f07-47d2-b46e-f505dadf6695
31542024-11-02T04:09:17.279ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_ae2b4e8c-8f07-47d2-b46e-f505dadf6695") failed: Property not found. retry in 103.476278ms file = illumos-utils/src/svc.rs:36 instance_id = 70aeba8a-4066-4620-bb46-a15fa016442a propolis_id = ae2b4e8c-8f07-47d2-b46e-f505dadf6695 zone = oxz_propolis-server_ae2b4e8c-8f07-47d2-b46e-f505dadf6695
31552024-11-02T04:09:17.361ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
31562024-11-02T04:09:17.362ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
31572024-11-02T04:09:17.363ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
31582024-11-02T04:09:17.364ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
31592024-11-02T04:09:17.366ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
31602024-11-02T04:09:17.373ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
31612024-11-02T04:09:17.374ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
31622024-11-02T04:09:17.374ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
31632024-11-02T04:09:17.375ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
31642024-11-02T04:09:17.376ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
31652024-11-02T04:09:17.377ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
31662024-11-02T04:09:17.382ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
31672024-11-02T04:09:17.383ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
31682024-11-02T04:09:17.403ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_ae2b4e8c-8f07-47d2-b46e-f505dadf6695") failed: Property not found. retry in 235.24203ms file = illumos-utils/src/svc.rs:36 instance_id = 70aeba8a-4066-4620-bb46-a15fa016442a propolis_id = ae2b4e8c-8f07-47d2-b46e-f505dadf6695 zone = oxz_propolis-server_ae2b4e8c-8f07-47d2-b46e-f505dadf6695
31692024-11-02T04:09:17.426ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
31702024-11-02T04:09:17.433ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
31712024-11-02T04:09:17.656ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_ae2b4e8c-8f07-47d2-b46e-f505dadf6695") failed: Property not found. retry in 557.114002ms file = illumos-utils/src/svc.rs:36 instance_id = 70aeba8a-4066-4620-bb46-a15fa016442a propolis_id = ae2b4e8c-8f07-47d2-b46e-f505dadf6695 zone = oxz_propolis-server_ae2b4e8c-8f07-47d2-b46e-f505dadf6695
31722024-11-02T04:09:18.232ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_ae2b4e8c-8f07-47d2-b46e-f505dadf6695") failed: Property not found. retry in 950.301793ms file = illumos-utils/src/svc.rs:36 instance_id = 70aeba8a-4066-4620-bb46-a15fa016442a propolis_id = ae2b4e8c-8f07-47d2-b46e-f505dadf6695 zone = oxz_propolis-server_ae2b4e8c-8f07-47d2-b46e-f505dadf6695
31732024-11-02T04:09:19.197ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_ae2b4e8c-8f07-47d2-b46e-f505dadf6695") failed: Property not found. retry in 764.939494ms file = illumos-utils/src/svc.rs:36 instance_id = 70aeba8a-4066-4620-bb46-a15fa016442a propolis_id = ae2b4e8c-8f07-47d2-b46e-f505dadf6695 zone = oxz_propolis-server_ae2b4e8c-8f07-47d2-b46e-f505dadf6695
31742024-11-02T04:09:19.975ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_ae2b4e8c-8f07-47d2-b46e-f505dadf6695") failed: Property not found. retry in 1.021092861s file = illumos-utils/src/svc.rs:36 instance_id = 70aeba8a-4066-4620-bb46-a15fa016442a propolis_id = ae2b4e8c-8f07-47d2-b46e-f505dadf6695 zone = oxz_propolis-server_ae2b4e8c-8f07-47d2-b46e-f505dadf6695
31752024-11-02T04:09:21.010ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_ae2b4e8c-8f07-47d2-b46e-f505dadf6695") failed: Property not found. retry in 580.585022ms file = illumos-utils/src/svc.rs:36 instance_id = 70aeba8a-4066-4620-bb46-a15fa016442a propolis_id = ae2b4e8c-8f07-47d2-b46e-f505dadf6695 zone = oxz_propolis-server_ae2b4e8c-8f07-47d2-b46e-f505dadf6695
31762024-11-02T04:09:21.602ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_ae2b4e8c-8f07-47d2-b46e-f505dadf6695") failed: Property not found. retry in 1.224991255s file = illumos-utils/src/svc.rs:36 instance_id = 70aeba8a-4066-4620-bb46-a15fa016442a propolis_id = ae2b4e8c-8f07-47d2-b46e-f505dadf6695 zone = oxz_propolis-server_ae2b4e8c-8f07-47d2-b46e-f505dadf6695
31772024-11-02T04:09:22.858ZINFOSledAgent (InstanceManager): Started propolis in zone: oxz_propolis-server_ae2b4e8c-8f07-47d2-b46e-f505dadf6695 file = sled-agent/src/instance.rs:1787 instance_id = 70aeba8a-4066-4620-bb46-a15fa016442a propolis_id = ae2b4e8c-8f07-47d2-b46e-f505dadf6695
31782024-11-02T04:09:22.866ZINFOSledAgent (InstanceManager): Propolis SMF service is online file = sled-agent/src/instance.rs:1796 instance_id = 70aeba8a-4066-4620-bb46-a15fa016442a propolis_id = ae2b4e8c-8f07-47d2-b46e-f505dadf6695
31792024-11-02T04:09:22.904ZINFOSledAgent (InstanceManager): Propolis HTTP server online file = sled-agent/src/instance.rs:1820 instance_id = 70aeba8a-4066-4620-bb46-a15fa016442a propolis_id = ae2b4e8c-8f07-47d2-b46e-f505dadf6695
31802024-11-02T04:09:23.042ZINFOSledAgent (InstanceManager): result of instance_ensure call is Ok(InstanceEnsureResponse { migrate: None }) file = sled-agent/src/instance.rs:932 instance_id = 70aeba8a-4066-4620-bb46-a15fa016442a propolis_id = ae2b4e8c-8f07-47d2-b46e-f505dadf6695
31812024-11-02T04:09:23.043ZINFOSledAgent: IGW mapping changed; external IPs refreshed file = sled-agent/src/sled_agent.rs:1233 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
31822024-11-02T04:09:23.043ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 7735643 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:57096 req_id = 7f0fdab9-66b3-4174-96bb-6afbb492a8a9 response_code = 204 uri = /eip-gateways
31832024-11-02T04:09:23.045ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Creating), migration_in: None, migration_out: None, time: 2024-11-02T04:09:23.045842974Z } file = sled-agent/src/instance.rs:780 instance_id = 70aeba8a-4066-4620-bb46-a15fa016442a propolis_id = ae2b4e8c-8f07-47d2-b46e-f505dadf6695
31842024-11-02T04:09:23.046ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:802 instance_id = 70aeba8a-4066-4620-bb46-a15fa016442a new_vmm_state = VmmRuntimeState { state: Starting, gen: Generation(3), time_updated: 2024-11-02T04:09:23.045842974Z } propolis_id = ae2b4e8c-8f07-47d2-b46e-f505dadf6695
31852024-11-02T04:09:23.046ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = 70aeba8a-4066-4620-bb46-a15fa016442a propolis_id = ae2b4e8c-8f07-47d2-b46e-f505dadf6695 state = SledVmmState { vmm_state: VmmRuntimeState { state: Starting, gen: Generation(3), time_updated: 2024-11-02T04:09:23.045842974Z }, migration_in: None, migration_out: None }
31862024-11-02T04:09:23.100ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Running), migration_in: None, migration_out: None, time: 2024-11-02T04:09:23.100503678Z } file = sled-agent/src/instance.rs:780 instance_id = 70aeba8a-4066-4620-bb46-a15fa016442a propolis_id = ae2b4e8c-8f07-47d2-b46e-f505dadf6695
31872024-11-02T04:09:23.100ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:802 instance_id = 70aeba8a-4066-4620-bb46-a15fa016442a new_vmm_state = VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2024-11-02T04:09:23.100503678Z } propolis_id = ae2b4e8c-8f07-47d2-b46e-f505dadf6695
31882024-11-02T04:09:23.100ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = 70aeba8a-4066-4620-bb46-a15fa016442a propolis_id = ae2b4e8c-8f07-47d2-b46e-f505dadf6695 state = SledVmmState { vmm_state: VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2024-11-02T04:09:23.100503678Z }, migration_in: None, migration_out: None }
31892024-11-02T04:09:23.126ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 424 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:57096 req_id = ec978845-694c-4bd3-8790-c75385656afd response_code = 204 uri = /vpc-routes
31902024-11-02T04:09:25.467ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
31912024-11-02T04:09:25.468ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
31922024-11-02T04:09:25.469ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
31932024-11-02T04:09:27.366ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
31942024-11-02T04:09:27.367ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
31952024-11-02T04:09:27.367ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
31962024-11-02T04:09:27.368ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
31972024-11-02T04:09:27.369ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
31982024-11-02T04:09:27.374ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
31992024-11-02T04:09:27.375ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
32002024-11-02T04:09:27.376ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
32012024-11-02T04:09:27.377ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
32022024-11-02T04:09:27.378ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
32032024-11-02T04:09:27.378ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
32042024-11-02T04:09:27.384ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
32052024-11-02T04:09:27.385ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
32062024-11-02T04:09:27.429ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
32072024-11-02T04:09:27.435ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
32082024-11-02T04:09:30.531ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:58938
32092024-11-02T04:09:30.531ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 107 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:58938 req_id = 67d4e831-0625-432c-a30b-599d1c965f46 response_code = 200 uri = /baseboard
32102024-11-02T04:09:32.596ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:37553
32112024-11-02T04:09:32.596ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 445 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:37553 req_id = 9e7736ee-aac4-4575-bc2d-f53118466815 response_code = 200 uri = /vmms/ae2b4e8c-8f07-47d2-b46e-f505dadf6695/state
32122024-11-02T04:09:32.596ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Stopping), migration_in: None, migration_out: None, time: 2024-11-02T04:09:32.596695427Z } file = sled-agent/src/instance.rs:780 instance_id = 70aeba8a-4066-4620-bb46-a15fa016442a propolis_id = ae2b4e8c-8f07-47d2-b46e-f505dadf6695
32132024-11-02T04:09:32.596ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:802 instance_id = 70aeba8a-4066-4620-bb46-a15fa016442a new_vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2024-11-02T04:09:32.596695427Z } propolis_id = ae2b4e8c-8f07-47d2-b46e-f505dadf6695
32142024-11-02T04:09:32.596ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = 70aeba8a-4066-4620-bb46-a15fa016442a propolis_id = ae2b4e8c-8f07-47d2-b46e-f505dadf6695 state = SledVmmState { vmm_state: VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2024-11-02T04:09:32.596695427Z }, migration_in: None, migration_out: None }
32152024-11-02T04:09:32.605ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Stopped), migration_in: None, migration_out: None, time: 2024-11-02T04:09:32.605407344Z } file = sled-agent/src/instance.rs:780 instance_id = 70aeba8a-4066-4620-bb46-a15fa016442a propolis_id = ae2b4e8c-8f07-47d2-b46e-f505dadf6695
32162024-11-02T04:09:32.605ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:802 instance_id = 70aeba8a-4066-4620-bb46-a15fa016442a new_vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(7), time_updated: 2024-11-02T04:09:32.605407344Z } propolis_id = ae2b4e8c-8f07-47d2-b46e-f505dadf6695
32172024-11-02T04:09:32.605ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = 70aeba8a-4066-4620-bb46-a15fa016442a propolis_id = ae2b4e8c-8f07-47d2-b46e-f505dadf6695 state = SledVmmState { vmm_state: VmmRuntimeState { state: Stopping, gen: Generation(7), time_updated: 2024-11-02T04:09:32.605407344Z }, migration_in: None, migration_out: None }
32182024-11-02T04:09:32.641ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Destroyed), migration_in: None, migration_out: None, time: 2024-11-02T04:09:32.641326925Z } file = sled-agent/src/instance.rs:780 instance_id = 70aeba8a-4066-4620-bb46-a15fa016442a propolis_id = ae2b4e8c-8f07-47d2-b46e-f505dadf6695
32192024-11-02T04:09:32.641ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:802 instance_id = 70aeba8a-4066-4620-bb46-a15fa016442a new_vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2024-11-02T04:09:32.641326925Z } propolis_id = ae2b4e8c-8f07-47d2-b46e-f505dadf6695
32202024-11-02T04:09:32.641ZINFOSledAgent (InstanceManager): terminating VMM that has exited file = sled-agent/src/instance.rs:816 instance_id = 70aeba8a-4066-4620-bb46-a15fa016442a propolis_id = ae2b4e8c-8f07-47d2-b46e-f505dadf6695
32212024-11-02T04:09:32.641ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
32222024-11-02T04:09:32.641ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:451
32232024-11-02T04:09:32.641ZINFOSledAgent (ZoneBundler): creating zone bundle context = ZoneBundleContext { storage_dirs: ["/pool/int/6308f1ec-c08e-4282-a1c2-db37d6b8a044/debug/bundle/zone", "/pool/int/fda73319-9c6d-44d8-9377-b69537374c59/debug/bundle/zone"], cause: TerminatedInstance, extra_log_dirs: ["/pool/ext/714116db-1daa-4c0c-b42f-cb5b471285ca/crypt/debug", "/pool/ext/68ab7f9e-24f1-481a-b99f-af893cdd74a3/crypt/debug", "/pool/ext/4846b5cc-cb2b-4eca-94f1-57e5f623f3d0/crypt/debug", "/pool/ext/61fda18b-b0fc-46cf-91c9-706f41acfc67/crypt/debug", "/pool/ext/09ce30cf-7836-438d-8cfe-4980ee894ac4/crypt/debug", "/pool/ext/39aca50d-85fd-4ecc-a443-45b121ccfb9a/crypt/debug", "/pool/ext/6b0637e8-562e-46b1-92a8-a7478ef38477/crypt/debug", "/pool/ext/010339bd-18fb-4f1c-9455-7caf9e67e152/crypt/debug", "/pool/ext/e3d9cf44-b37a-4529-b1ca-03b2819010d3/crypt/debug"] } file = sled-agent/src/zone_bundle.rs:339 zone_name = oxz_propolis-server_ae2b4e8c-8f07-47d2-b46e-f505dadf6695
32242024-11-02T04:09:33.280ZINFOSledAgent (ZoneBundler): finished zone bundle file = sled-agent/src/zone_bundle.rs:1142 metadata = ZoneBundleMetadata { id: ZoneBundleId { zone_name: "oxz_propolis-server_ae2b4e8c-8f07-47d2-b46e-f505dadf6695", bundle_id: c793a465-fbf0-497c-83fe-710183e2eced }, time_created: 2024-11-02T04:09:32.642116342Z, version: 0, cause: TerminatedInstance }
32252024-11-02T04:09:33.280ZWARNSledAgent (InstanceManager): Halting and removing zone: oxz_propolis-server_ae2b4e8c-8f07-47d2-b46e-f505dadf6695 file = sled-agent/src/instance.rs:1032 instance_id = 70aeba8a-4066-4620-bb46-a15fa016442a propolis_id = ae2b4e8c-8f07-47d2-b46e-f505dadf6695
32262024-11-02T04:09:35.054ZINFOSledAgent (InstanceManager): halt_and_remove_logged: Previous zone state: Running file = illumos-utils/src/zone.rs:279 instance_id = 70aeba8a-4066-4620-bb46-a15fa016442a propolis_id = ae2b4e8c-8f07-47d2-b46e-f505dadf6695
32272024-11-02T04:09:35.072ZINFOSledAgent (PortManager): Removed route set for subnet file = illumos-utils/src/opte/port_manager.rs:964 id = RouterId { vni: Vni(5808556), kind: System }
32282024-11-02T04:09:35.072ZINFOSledAgent (PortManager): Removed route set for subnet file = illumos-utils/src/opte/port_manager.rs:964 id = RouterId { vni: Vni(5808556), kind: Custom(V4(Ipv4Net { addr: 172.30.0.0, width: 22 })) }
32292024-11-02T04:09:35.072ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = 70aeba8a-4066-4620-bb46-a15fa016442a propolis_id = ae2b4e8c-8f07-47d2-b46e-f505dadf6695 state = SledVmmState { vmm_state: VmmRuntimeState { state: Destroyed, gen: Generation(9), time_updated: 2024-11-02T04:09:35.072684931Z }, migration_in: None, migration_out: None }
32302024-11-02T04:09:35.134ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = 70aeba8a-4066-4620-bb46-a15fa016442a propolis_id = ae2b4e8c-8f07-47d2-b46e-f505dadf6695 state = SledVmmState { vmm_state: VmmRuntimeState { state: Destroyed, gen: Generation(9), time_updated: 2024-11-02T04:09:35.072684931Z }, migration_in: None, migration_out: None }
32312024-11-02T04:09:35.138ZINFOSledAgent (InstanceManager): State monitoring task complete file = sled-agent/src/instance.rs:969 instance_id = 70aeba8a-4066-4620-bb46-a15fa016442a propolis_id = ae2b4e8c-8f07-47d2-b46e-f505dadf6695
32322024-11-02T04:09:35.471ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
32332024-11-02T04:09:35.472ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
32342024-11-02T04:09:35.472ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
32352024-11-02T04:09:35.612ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:53372
32362024-11-02T04:09:35.612ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 58 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:53372 req_id = f9d4e9cf-b324-456a-b83f-fd14dccd5d97 response_code = 200 uri = /vpc-routes
32372024-11-02T04:09:35.626ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
32382024-11-02T04:09:35.626ZINFOSledAgent: IGW mapping changed; external IPs refreshed file = sled-agent/src/sled_agent.rs:1233 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
32392024-11-02T04:09:35.626ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 95 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:53372 req_id = a4ad74f0-188b-48f2-ab1f-a7ece06e60cd response_code = 204 uri = /eip-gateways
32402024-11-02T04:09:35.635ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 230 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:53372 req_id = 264caae2-898b-46c1-aa24-6ed48a3b7570 response_code = 204 uri = /vpc-routes
32412024-11-02T04:09:35.713ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:63571
32422024-11-02T04:09:35.713ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 129 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:63571 req_id = dfc3e16f-b076-4e4a-8fee-6a6399d54896 response_code = 200 uri = /v2p
32432024-11-02T04:09:36.899ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:47297
32442024-11-02T04:09:36.899ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 170 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:47297 req_id = 5f02a44c-bcf3-4239-b162-5fb2f8d7785c response_code = 200 uri = /v2p
32452024-11-02T04:09:37.072ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:37567
32462024-11-02T04:09:37.072ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 85 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:37567 req_id = 43cefcbd-72a4-496e-96b3-c49be8405d8d response_code = 200 uri = /vpc-routes
32472024-11-02T04:09:37.087ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
32482024-11-02T04:09:37.087ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 78 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:37567 req_id = 6ecf16bd-bffc-4994-a515-ade5dd490cd9 response_code = 204 uri = /eip-gateways
32492024-11-02T04:09:37.095ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 208 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:37567 req_id = 97b8200e-855d-4506-a87d-ba0c5152bf25 response_code = 204 uri = /vpc-routes
32502024-11-02T04:09:37.370ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
32512024-11-02T04:09:37.371ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
32522024-11-02T04:09:37.372ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
32532024-11-02T04:09:37.372ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
32542024-11-02T04:09:37.373ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
32552024-11-02T04:09:37.376ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
32562024-11-02T04:09:37.377ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
32572024-11-02T04:09:37.377ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
32582024-11-02T04:09:37.379ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
32592024-11-02T04:09:37.379ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
32602024-11-02T04:09:37.380ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
32612024-11-02T04:09:37.386ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
32622024-11-02T04:09:37.386ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
32632024-11-02T04:09:37.430ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
32642024-11-02T04:09:37.437ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060
32652024-11-02T04:09:37.660ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:47907
32662024-11-02T04:09:37.660ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 39 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:47907 req_id = ccf539cd-e071-4ef4-b176-9df699f6df48 response_code = 200 uri = /vpc-routes
32672024-11-02T04:09:37.674ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1207 sled_id = 5b8cdd5a-f021-45f2-8323-b6c8f9776060 values = ExternalIpGatewayMap { mappings: {} }
32682024-11-02T04:09:37.674ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 35 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:47907 req_id = 3414538d-c9bd-4731-b856-e56b049cbd90 response_code = 204 uri = /eip-gateways
32692024-11-02T04:09:37.681ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 144 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:47907 req_id = 4546286c-14bc-4754-aa7e-8c26bc6dcd9f response_code = 204 uri = /vpc-routes